StreetScouting is a platform developed by DataScouting for the GRUBLES project that uses state-of-the-art Machine Vision methods to automatically detect and geotag urban features from street-level data
Urban environments are evolving rapidly in big cities; keeping track of these changes is becoming harder and harder. Updated information regarding urban features in a particular region can be crucial for tasks like predicting population growth, analyzing park space, surveying water supply, identifying transportation patterns or recognizing food supply demands.
Counting and geotagging the number of particular features in a given urban area or city is crucial when making decisions for government policies and administration. Urban planning officials and stakeholders often lack updated information regarding such features and that can potentially lead to wrong decisions. The traditional methods used for counting and geotagging urban features, include, in most cases, manual inspection and consequently are time-consuming, costly, and labor-intensive. Therefore, there is an increasing need for an automated approach to replace conventional visual inspection.
GRUBLES is a project developed by DataScouting and Infodim to improve the collection of landscape and socioeconomic indicators for smart cities. GRUBLES is the Norwegian word for “think” but breaking it down it is more than that: in fact, it stands for GeoReferenced UrBan LandscapE and Socioeconomic Indicators using Machine Vision. The project aspires to utilize primary indicators in order to perform a statistical study to find correlations with some socioeconomic indicators of the region.
“May I start on how grateful I feel for being part of the GRUBLES Project. I appreciate that I had this chance to work with and mentor great people within the Front-End Software Engineering Department. The Project itself is an amazing idea and I look forward to people using GRUBLES to gather meaningful information just from uploading a video. Combining the departments of Machine Learning and Software Engineering gave us the opportunity to develop a platform tailored to offer a joyful experience and make advanced technologies simple to use. It was a pleasure working with the rest of the team, they would always go the extra mile in order to achieve challenging results. I myself, also learned a lot within this journey and improved my mentoring skills,” says Ioakeim James Theologou, Front End Software Engineering Mentor.
StreetScouting: A Deep Learning Platform for Automatic Detection and Geotagging of Urban Features from Street-Level Images
For this, DataScouting developed StreetScouting, an extensible fully featured platform that applies deep learning and computer vision techniques to automatically detect and extract urban features in cities. StreetScouting’s main task is to process GPS and video data of a particular region and output visualizations and data regarding various objects and urban features. Objects, such as trees, trash and recycle bins, street light poles and lamps, vehicles, traffic lights, traffic signs, and other features, such as the width of the road or the existence of sidewalks are some of the features that can be detected and localized by the platform. You can find more details in the related publication in Applied Sciences journal.
“This project represents an innovative approach to understanding and mapping urban environments, and I am proud to be a part of it. One of the key contributions of the R&D department, which I was part of, is the project methodology. We utilized state-of-the-art deep learning techniques to accurately detect and classify urban features from street-level images. This methodology has been thoroughly validated through extensive experimentation and formal analysis, ensuring high levels of accuracy and reliability. Another important contribution of StreetScouting is the developed software services, allowing the machine learning models to provide user-friendly and easy to navigate results. Another significant contribution is the data curation. StreetScouting uses a vast and diverse dataset of street-level images, which we carefully curated to ensure a high level of representativeness and accuracy. This dataset forms the foundation of the platform, and its quality is crucial to the success of the project. Lastly, the R&D department has conducted a formal analysis of the platform, investigating its performance and limitations. This investigation not only helps to improve the platform but also provides valuable insights for future research in this area. I would like to extend my sincerest gratitude to all the collaborators who have contributed to this project. Without their hard work and dedication, StreetScouting would not have been possible. I look forward to continuing to work together and pushing the boundaries of what is possible with deep learning and street-level images,” says Sotirios Moschos, Machine Learning Engineer.
StreetScouting: brief description
StreetScouting is able to detect and track urban features, reducing the gap between a fully automatic and manual detection approach. To achieve this, StreetScouting incorporates many state-of-the-art computer vision approaches for object detection, semantic segmentation, multiple object tracking, depth estimation, and others. It provides an organized and easy-to-use environment that can be utilized by urban planning administrators and policymakers, municipal offices, geoinformatics companies, and other commercial organizations for applications related to public health, real estate, traffic detection, waste management and others. StreetScouting generated data can be further used for statistical analysis and geographic information systems. It has the ability to generate so much information with minimal user input. It offers a secure environment for users to explore, but on the other hand it has the ability to share information within its users. The extent of its capabilities does not stop there, because as a user you can use the produced results as you want, downloading a file with the complete results of a video containing all the detected objects and their metadata. One of StreetScouting’s core features is the ability to display all the detected objects within a video using an Open Street Map, having the option to download the video frame in which the object was detected. Each displayed object class has its own custom icon, and there is also an option to filter the displayed objects based on their class.
StreetScouting: Platform Architecture and User Interface
The StreetScouting platform consists of five modules that communicate with each other. The proposed platform is currently running as a web application on multiple servers and consists of a microservice architecture that enables portability and scalability.
“Working on an innovative project like StreetScouting and with the team and mentors that I had, is a great opportunity to develop and improve my overall skills. All its features and their challenges, kept all the team members excited to work with it and break the barriers. After all this hard work analyzing, developing and testing the whole platform, we are delighted with the result that we’ve produced. Being able to generate so much information about an urban area using the slightest input is mind-blowing. We’ve used the latest and most reliable technologies that we could, in order to provide to the users of this platform a stable and secure environment to explore. We have also tried to think ahead of the time while developing the platform creating clean, reusable and feature proof code, so that the next version of it will be able to have a strong foundation to build upon it. Overall, it was a great experience to work with it and I can’t wait to see its next stop,” says Archontis Pipertzis, Software Engineer.
StreetScouting will continue to obtain updates and keep up with the latest trends in deep learning research. The team has also released a publicly available dataset that can further increase the impact of the project and assist research efforts in the field.
“This project is a clear example of how research deliverables can be transformed into successful commercial products when an extremely competent team is asked to combine their state-of-the-art research with robust software engineering,“ says Stavros Doropoulos, Chief Information Officer.
Special thanks to Polychronis Charitidis, Michalis Michailidis, Iordanis Kostelidis and Christos Diou for providing their invaluable contribution to this project.
The GRUBLES project is materialized under the Action “RESEARCH – CREATE – INNOVATE”, which is co-financed by National (Operational Programme Competitiveness, Entrepreneurship and Innovation 2014-2020-EPAnEK, National Strategic Reference Framework-NSRF) and EU Funds (European Union, European Regional Development Fund).
