Yuhei Yamamoto

In Japan, the Sport Basic Plan was formulated in 2012, which mentions not only the development of highly qualified sports instructors but also new perspectives such as the provision of programs in which everyone can enjoy the value of sports together. Against this background, trials of new technologies utilizing the latest ICT equipment such as sensor devices are being made. However, in college sports, where financial resources are often limited, shooting with hand-held video cameras is the most common method, which may cause the manager to overlook important scenes of play depending on his or her skill level. This problem affects not only competitors but also spectators. To solve this problem, we develop a system for capturing video images of the entire field with multiple fixed video cameras so that the system can automatically switch from one to another video of the most appropriate camera for the respective scenes of play. The results of the demonstration experiments in basketball and futsal showed that the switching video of the proposed technology can be utilized for tactical analysis in sports.

In our country, AI-based survey methods are being promoted to streamline traffic volume surveys. Existing research has shown that vehicle section identification using AI can count sectional traffic volumes by vehicle type, but accuracy decreases due to flares and scenery reflections. Additionally, vehicle types can be determined by recognizing the leading number of the classification code on license plates, but the recognition accuracy decreases when the characters are unclear. Therefore, this study proposes a method to determine vehicle types by recognizing classification codes when the characters are clear and by using vehicle section identification results when the characters are unclear. Applying the proposed method to videos taken at three different locations resulted in a high accuracy with an F-measure of 0.95 or higher at all locations. In the future, we aim for early practical application in automotive traffic volume surveys by accommodating license plates with designs.

Recently, there have been many cases of analyzing traffic volume and travel speed per route using probe data collected from automobiles. In addition, if the behavior of individual vehicles could be understood in terms of driving lanes, more advanced road traffic analysis could be realized, such as analyzing the behavior of lane changes at mergers and crossings. In this study, we devised a method to estimate the lane of driving of individual vehicles from ETC2.0 probe data, which has higher positioning accuracy than ETC2.0 probe data, by superimposing the driving trajectories of individual vehicles on a mesh divided by lane. Then, machine learning was applied to the time-series changes in the unique numbers assigned to the mesh. The results suggest that although the estimation results are affected by the positioning accuracy of ETC2.0 probe data, it is possible to estimate the driving lane of an individual vehicle with higher accuracy than when the positioning points are checked one by one.

In Japan, AI is being incorporated into automobile traffic volume surveys to enhance efficiency. By recognizing license plates in these surveys, it's possible to measure vehicle flow, refining the accuracy of the survey. However, when the video footage is taken from a high vantage point on the road's shoulder, the license plates appear tilted. To correct this, the image needs to be adjusted to a frontal orientation. If the license plate and the vehicle have the same color, determining a reference point for the projective transformation becomes challenging. To address this issue, we developed a method that corrects the license plate to a front-facing orientation using a projective transformation, based on the position of the fourth digit of the serial number—a consistent feature on all license plates. Our experimental results indicated successful character extraction from 479 out of 500 images. We aim to apply this technique in future traffic flow studies to identify the same vehicle at multiple locations.

Counting the number of people in a crowd at an event or during a disaster is important to prevent accidents from occurring. In recent years, various methods have been developed to count the number of people using deep learning. However, it is difficult to automatically select and apply the most appropriate counting method, because video images of crowds are taken under various conditions. In this study, we focused on the differences in the trends of the detection results of head counting methods and devised a method to estimate the number of people using regression analysis. As a result of the verification, it was found that the regression analysis can reduce extreme false positives and omissions and estimate the number of people more accurately by complementing the weaknesses of each head counting method.

In current pedestrian traffic volume surveys, pedestrians passing through the surveyed cross section are generally counted manually, which limits the survey days and times. In recent years, there has been an increase in the number of surveys using video images taken by video cameras, but personal information and privacy must be taken into consideration. Therefore, LiDAR, which can measure the target pedestrian as a set of three-dimensional coordinate points, has been attracting attention. However, repetitive LiDAR cannot measure the measurement range exhaustively and is difficult to be applied to the survey. In this study, we conducted a pedestrian traffic survey using deep learning with point cloud data measured by non-iterative LiDAR, which can measure the measurement range exhaustively. The results of pedestrian counting showed that the correct response rate was 67.2% in the low case and 84.7% in the high case, indicating that the point cloud data measured by non-repeatable LiDAR has potential to be applied to pedestrian traffic volume surveys.

When subject to excessive loads in the context of excavation construction, earth retaining walls can undergo deformation that is challenging to assess visually, potentially resulting in accidents involving wall collapses. Hence, previous research has explored installing sensors within the retaining walls and using surface targets in conjunction with TS to measure deformation quantitatively. Nevertheless, these methods have presented significant challenges, including high costs, extensive labor requirements, and limitations in comprehensively assessing deformation. Therefore, we focused on cost-effective LiDAR technology, which offers the capability to provide real-time, comprehensive 3D data for measurement targets. We proposed a method for detecting deformations in earth retaining walls using LiDAR, representing a non-repetitive measurement approach. The experimental results demonstrated the effectiveness of the proposed method in detecting surface deformations in earth retaining walls. In the future, our objectives encompass applying this method to practice construction sites, identifying and resolving implementation challenges, and refining the proposed technique.

With progress of Web production technology, websites are being mass-produced every day all over the world. In the context of roadside stations gaining attention as hubs for regional revitalization, there is a current reevaluation of the creation and operation of websites for each roadside station, addressing the challenge of insufficiently showcasing the diverse charms of different regions. As users commonly browse websites with higher search rankings in web searches, it becomes crucial to ensure that the web-sites of roadside stations rank prominently in search results. Therefore, it is difficult to formulate coun-termeasure policy for own website. In this research, we propose a present method to evaluate SEO measures items from relation to search ranking by analyzing many applicated web pages comprehen-sively. These web pages retrieved from web search engine using sets of search queries related to main keywords of website.

In vehicle traffic surveys, methods are being proposed to replace the current visual surveys with AI traffic surveys. These AI traffic surveys are utilizing advanced image processing methods more than ever before, and against this background, new vehicle-count methods are being proposed. These methods analyze 4K videos taken by video cameras at survey points, and if these existing methods could be applied to existing CCTV images owned by road managers, it would become possible to conduct AI traffic surveys without the necessity of installing new cameras. However, many CCTV videos have a low resolution because they are intended for long-term monitoring, and in addition, there is a dullness due to image distortion or deterioration due to the long length of time since installation. Therefore, there is a high possibility that the accuracy may be insufficient for traffic volume observation using existing methods. In this research, focusing on the fact that features due to image distortion and dullness can be removed by blurring vehicle images detected from the entire image, a vehicle-type recognition method is proposed using only major features of the vehicles. In a demonstration experiment, we compared the classification accuracy before and after the introduction of the proposed method, and showed that it is possible to count small and large vehicles with higher accuracy than the existing method for SD and HD quality video images. As a result, we were able to confirm the effectiveness of the proposed method in AI traffic volume surveys.

In Japan, road traffic censuses are conducted to assess road traffic conditions. Recently, techniques for counting traffic volume from video images have been attracting considerable attention in order to improve work efficiency and save labor, and a large number of technologies have been developed. However, since traffic volume surveys are often conducted 24 hours a day, day and night, at various sites and under various weather conditions, existing technologies have yet to reach the counting accuracy required in practice. The authors aim to develop techniques for traffic volume surveys applicable in practice by applying artificial intelligence. This paper reports the results of a case study in which the proposed techniques were applied to the video taken during actual traffic volume surveys.

In its i-Construction policy, the Ministry of Land, Infrastructure, Transport and Tourism has stipulated a manual for public surveying using by UAV photogrammetry with the aim of improving productivity at construction sites. However, UAV photogrammetry requires a huge amount of time to generate point cloud data from photographs, causing a problem that it is difficult to monitor the daily progress of the construction site. There is another problem that it is incapable of taking measurements except daylight hours. Against this backdrop, we have been developing laser scanner units for UAV equipped with LiDAR, IMU and GNSS receiver. Then, we clarified the error factors that are expected to affect the precision prescribed of point cloud data and summarized the requirements for the onboard equipment and the method of generating point cloud data. However, we have not yet proposed nor developed the laser scanner units as well as for generating point cloud data considering these requirements. Thus, in this paper, we developed the laser scanner units and a new method for generating point cloud data within the laser scanner units to optimize the error factors based on these requirements clarified in the existing study.

The road administrator routinely inspects pavements, however, the need for repair depends on the inspector's experience and knowledge, and the lack of skilled inspectors has made improving the efficiency of inspection work an urgent issue. As a solution to this problem, previous studies have diagnosed pavement damage during driving by using probe data that record the driving position and acceleration of vehicles. On the other hand, if probe data can be collected at the same time as daily inspections, it will be possible to identify damaged areas through damage diagnosis, analyze data trends over multiple time periods to understand the progress of damage, and predict future damage.  In this study, we attempt to apply LSTM, which is good at classifying and predicting time-series data, to the extraction of damage locations and damage prediction. As a first step, we constructed a pavement damage diagnosis model using LSTM and confirmed its usefulness with a damage extraction rate of about 80%.

In our country automobile traffic census is generally conducted manually. However, the shortage of surveyors is becoming the norm due to the declining birthrate and aging population, and there is a need to make surveys more efficient and labor-saving. As a solution to this problem, AI-based traffic census has been attracting attention. But it is impossible to identify the same vehicle because most of the technology target cross-sectional traffic volume measurement. Therefore, it is thought that the same vehicle can be identified by recognizing the letters on the license plate. In this study, first, we devised a method for extracting and recognizing a series of designated license plate numbers and verified its accuracy. The devised method was then applied to video images taken between the two locations to verify whether the same vehicle could be identified. As a result, revealed the possibility of identifying identical vehicles, which could not be measured with existing technology.

In traffic census, it is expected to develop image processing technologies for counting number of passing automobiles by analyzing video image. Many counting technologies using deep learning have been proposed. It is difficult to maintain sufficient accuracy because new automobiles are sold year after year. Therefore, it is necessary to maintain high accuracy by re-learning training data of automobiles with new shapes and colors continuously. However, maintenance labor cost is huge because training data have to be created continuously. In this research, technique to recursive active learning for segmentation of automobile parts is proposed and clarified its usefulness.

In our country, the utilization of ICT is promoted toward streamlining and labor saving in the investigation of the traffic volume of automobiles. In particular, investigation methods using moving images are drawing attention and various methods have been proposed. Now, the counting of a spot traffic volume has been realized but the flow investigation of vehicles between points has not been realized. Therefore, by adding a technology to identify the same vehicle to the existing technology, the investigation of the flow of vehicles between points becomes possible and this contributes to the sophistication of the investigation. Accordingly, this research created a method to identify the same vehicle by recognizing a series of designated numbers on license plates from moving images for a traffic volume investigation shot with more than one wearable camera. In the experiment, the created method was applied to moving images shot between two points to verify whether the same vehicle could be identified. As a result, the possibility was clarified that the same vehicle could be identified, which could not be measured with existing technology.

It is important to count the number of people in a crowd during an event and commuting hours to prevent accidents. In recent years, a method has been developed to count the number of people easily from images by improving the speed and accuracy of deep learning. However, since crowd shooting conditions such as the installation angle and height of a camera are varied, depending on the size of a person and the degree of occlusion in a moving image, also how the person seems is varied. Thus, it is difficult to accurately count the number of people in a crowd under various conditions using one counting method. Consequently, it was considered that a counting method could be established to secure a certain degree of accuracy by categorizing scenes and conditions for shooting a crowd and changing a method to count the number of people best for the status of the crowd in each scene and shooting condition as necessary. In this research, four types of methods to count the number of people were applied to each scene to clarify the best method for each scene and problems toward the changing of an applied method.

For safe, secure, and smooth road space maintenance, it is necessary to understand traffic reality changing from moment to moment. Therefore, road administrators analyze road traffic using probe data acquired from vehicles. As a concrete example, after map matching processing for identifying a road where probe data have been acquired using road network data in which the road is represented as a line segment, a travel speed in each road unit, etc., are analyzed exhaustively. Meanwhile, in the analysis of a speed for each movement direction for each intersection, etc., map matching and counting often cannot be processed mechanically. This research created a method to determine the movement directions of probe data at an intersection using polygon meshes for understanding a positional relationship only with the latitude and longitude. Accordingly, a demonstration experiment was conducted, the percentage of correct answers for movement directions was 90% or more and knowledge was obtained that movement directions can be determined mechanically using the created method.

Research on tracking and performance analysis of athletes using video images has been actively conducted with the aim of improving athletes' competitive performance. However, when filming plays in field sports, it is difficult to capture the entire field with a single camera without filming from a specific point, such as a spectator's seat on the corner side, because the field is long sideways. Even if the entire field is captured, the players at the back of the field appear small, making analysis difficult. Other issues include the fact that since many coaches and analysts film where the play is progressing, it is hard for them to track the ball seamlessly when the position of play changes significantly depending on the position of ball. To solve this problem, we develop in this study a technology to automatically generate panoramic video images that cover the entire field by using two video cameras. Using this technology, we aim to generate panoramic images of the entire field that makes it possible to surely measure and analyze all players and the ball.

The authors have aimed at detecting and tracking players using video cameras in field sports. However, there is a problem that the detection accuracy is lowered at the place where players have an occlusion. Deep learning is a solution to this problem, but it requires the manual creation of large amounts of training data which is costly. In this research, the authors propose a method for automatically generating training data to build a detector that can identify players with high accuracy for field sports, and a method for updating this detector to input it to various videos.

In Japan, road administrators perform traffic censuses to understand the status of automobile traffic. Recently, in these censuses, techniques for counting the traffic volume from video images have attracted attention for the purpose of improving work efficiency and to save labor. These techniques can count with practical accuracy in video images taken in the daytime. However, the counting accuracy is reduced in video images taken in the nighttime as sufficient brightness are not secured and a shape and color of the vehicle are obscured. In this research, we develop a traffic census technique for application to nighttime traffic using existing techniques. This technique converts video images shooted the nighttime into video images taken in the daytime using deep learning. Furthermore, we clarified the usefulness of the proposed technique through a demonstration experiment.

Currently, in many cases of pedestrian traffic measurement, pedestrians are counted by field surveying or a visual check of motion images. Accordingly, there are many problems including error counts caused by human error and the risk of heatstroke caused by outdoor measurement for long hours. In addition, in the environment where unspecified large numbers of people come and go, a problem of a decrease in pedestrian measurement precision due to occlusion has been revealed, and there is no solution established for this problem yet.  In this study, a practical method for pedestrian traffic measurement was derived by verifying the technology of person recognition using deep learning capable of treating occlusion, and clarifying its problems. As a result, it was found that even if most of the person region is hidden, the number of pedestrians can be measured with high precision by applying the proposed method.

In Japan, owing to the steadily declining working population, innovative technologies that improve productivity in construction sites using i-Construction are urgently required. This necessitates the tracking of not only construction vehicles but also workers using technologies such as AI and IoT. Against this background, many technologies that were able to track both workers and construction equipment on construction sites had been developed. However, challenging subjects for automating tracking tasks remained as those technologies were applied in sections where extended and heavy occlusion occurs frequently. Therefore, an identification method that matches appearance features of workers and construction vehicles before and after occlusion is proposed in this research, thereby contributing to the development of tracking technologies that enable successive tracking through occlusion sections. Experiments conducted using videos from actual construction sites showed that the method was capable of restoring tracking progress in occlusion sections by identifying each worker and construction vehicles individually before and after each section.

In recent years, effective safety management measures based on the Internet of Things (IoT) have been established to prevent industrial accidents in construction sites. In response to these efforts, technologies that use object detection methods to obtain the location information of workers and construction vehicles can contribute toward safety management. However, it is difficult to detect workers and construction vechiles from video images by object detection methods in construction site where dangerous areas are constantly changing due to their coexistence. Therefore, the application of deep learning can be considered, but in order to detect them accurately, it is necessary to update the detection model specialized for construction sites. However, it takes a lot of cost to manually create a training data for constructing model. Therefore, in this research, we propose an automatic generation method of training data for deep learning to realize high-precision for detecting workers and construction vehicles in construction sites. Experiment was performed with application of the proposed system to the video data at construction site, and its usefulness was confirmed.

In Japan, road administrators perform traffic censuses to understand the status of automobile traffic. In this census, it is common for investigators to visually check the automobiles and count the numbers of passing automobiles for both small and large cars. But, it is difficult to secure workers because the working age population is decreasing. Therefore, the Ministry of Land, Infrastructure, Transport and Tourism will abolish the manual survey and consider introducing some techniques that automatically count the number of passing automobiles using video images. In existing research, techniques for the classification of automobile type have been developed using machine learning or deep learning. These techniques are not yet accurate enough to be used in practice. In this research, we develop a technique to count the number of passing automobiles for each automobile type. This technique classifies the automobile type based on the outer shape of the automobile and the shape of the parts, which investigaters pay attention to when they classify the automobile type manually. Furthermore, we clarified the usefulness of the proposed technique through some demonstration experiments.

In field sports, to realize tactical analysis, many techniques for athlete detection, identification and tracking by analyzing videos have been developed. However, many techniques have the same problems that the accuracy of detection, identification and tracking will decline in quality while athletes are occluding. In this research, the authors focus on the fact that the features of each athlete before and after occlusion are approximated, and then developed the method for identifying athletes with their features before and after occlusion. In the results, we were able to solve the problem of degrading detection, identification and tracking accuracy in the occlusion interval.

In recent years, the utilization of ICT for field sports has been promoted in Japan. And then, a lot of researches have been conducted on tactical analysis using positions and trajectories of players obtained from play video data. However, in many of them, an area of an athlete may be misidentified as area containing shadow or area of only shadow, which leads to a problem of acquiring accurate positions of athletes. Therefore, in this research, a method to remove the shadows of players from soccer play videos by using pix2pix, a type of GAN, to generate shadow-free videos is proposed and implemented. As a result, it was verified through experiments that the shadows of athletes could be removed accurately. Furthermore, the effectiveness of the proposed method was also confirmed by tracking accuracy comparison of an existing person tracking method with and without shadow removal.

When designing a soil structure, results obtained from a soil investigation are converted into parameters that are directly needed for design calculation; a conversion error in such a case, however, is considered as a challenge to be addressed. If a soil constant necessary for numerical analysis of soil can be estimated accurately by using AI, the quantity of information required for analysis is expected to be increased. In this study, Internal friction angle data obtained by triaxial compression tests were arranged from KANSAI Soil databases and a trial estimation was made on values for Internal friction angle by means of artificial neural networks. First, 504 data sets collected from the Kansai area soil information databases (data of Kobe City) were created. As a result of the above, the coefficient of determination became 0.657, showing a Internal friction angle by excluding the UU test. Regarding future prospects, increase in AI data, comparison of AI algorithms, estimation of other soil constants, and verification of applicability per region are scheduled to be conducted.

Recently, countermeasures against traffic congestion and accidents have been promoted by utilizing probe data, which enables us to grasp the driving history and behavior of individual vehicles. Currently, the main application of probe data is to analyze traffic volume and travel speed per route, but we believe that more advanced road traffic analysis can be realized if probe data can be applied to micro traffic analysis at the driving lane level. Therefore, in this study, in order to estimate the driving lane using probe data, we analyzed the characteristics of the data and devised a method to estimate the driving lane and whether or not a lane change has occurred. As a result, we confirmed that lane change points can be estimated with high accuracy by applying a machine learning model that focuses on changes in acceleration. Furthermore, we found that combining the results of both driving lane and lane change estimation results provides more reliable estimation of the driving lane.

Under the policy of Society 5.0, all kinds of information and intellectual information across domains are extracted, exchanged, collaborated, shared, and reused. In particular, the SNS (Social Networking Service) has been used in a wide range of generations, researches on the data science has been actively conducted in fields. And then, the most important thing is the reliability, it is important to acquire attribute such as the gender, age, occupation, and region of the poster. The authors have been able to obtain certain results in the acquisition of three attributes other than the area attribute. However, it was difficult to estimate the locality because there were few characteristic words and the rate of location information was low. In the present research, we focus on characteristic words and words that express the same thing, and words that are used differently in each region. It is the dialects like “Kaitenyaki, Imagawayaki and Obanyaki”. Then, we propose a method to gradually extend the regional estimation model by automatically extracting users in the target region from the profile fields of SNS and learning them sequentially. In this way, we were able to estimate the data more accurately than existing methods.

In Japan, policies have been positively promoted aiming at securing sport opportunities for every citizen, realization of a society of health and longevity, regional vitalization through sports, and improvement in athletic ability at sports towards the Tokyo 2020 Olympics and Paralympics, with Japan Sports Agency as a flag-bearer. Among others, an effort of grasping the ability of athletes or game analysis using the state-of-the-art sensors or wearable terminals is one of the important policies of Ministry of Internal Affairs and Communications, which attracts high attention. Japan women's national volleyball team achieved magnificent results such as winning a medal in the Olympics by utilizing tablet terminals and the game analysis software in selecting the members for the world championships or commands during the games. However, to apply ICT to the sport area, it is necessary to get hold of not only personnel who have knowledge about handling sensor equipment and expert knowledge of the game but also excellent data analysts. Therefore, application of ICT to sports is limited to part of professional sports. Its introduction into amateur sports faces a mountain of problems. In this study, with focus on volleyball, which is an indoor sport, we make research on the trends of studies on game analysis, and propose a system for supporting decision making of games by recognizing multiple athletes and balls from the images of games shot with video cameras using deep learning. We confirmed that it is possible to generate analysis codes for inputting on Data Volley with proposed method. Data Volley is used by national teams all over the world and college sports.

In the field of sports engineering, though policies for developing human resources in sports are promoted nationally, lack of managers and coaches who can give highly-qualified instruction to an individual player is an urgent problem. Therefore, a swing measurement unit is developed for quantitatively analyzing a batter’s swing form in baseball. However, it is difficult for managers and coaches to utilize the sensor data and give players an instruction based on the data. The purpose of the study is to find out new knowledge for giving appropriate advice to individual players by analyzing the swing characteristics with the swing measurement unit. In order to verify whether the swing characteristics of university baseball players can be properly classified, the swing data and batting performance data were classified individually using the k-means method. As a result, when the swing characteristics and batting performance were classified into two cluster by the k-means method, the players could be classified with high accuracy. Furthermore, the principal component analysis was performed to examine the relationship between the measurement results obtained from the swing measurement device and the swing characteristics in conjunction with this classification result. The results of principal component analysis also showed that it was possible to classify high-ranked players and low-ranked players. From these results, it showed elements of the measurement items related to classification. It was found that players with long swing rotational radius, short swing time and high head speed were classified as high-ranked players.

In Japan, towards the Tokyo Olympics and Paralympics, the Japan Sports Agency has been founded, and policies related to training athletes and improving competitiveness are being promoted. To achieve these policies, many researches use AI technology to analyze sports data gathered by video cameras and IoT devices. To make effective use of those data, it is necessary to manage them with a unified time stamp. However, it is difficult to unify the time stamp because these devices are not linked to each other. To solve this problem, the time synchronization is often adopted manually, but it costs a lot of manpower. Therefore, in this research, we firstly organize the problems of time synchronization between video cameras and between video cameras and GNSS device by investigating existing researches. And then, we propose an efficient and highly accurate time synchronization technologies that can solve these problems.

In recent years, as measurement devices have advanced due to sensor and information technology, we have been able to measure bat swing data just after baseball impact. Therefore, the purpose of this study was to examine the characteristics of a batter’s swing using batting skill assessments from baseball coaches with significant experience. Finally, the practicality and effectiveness of baseball coaching methods, particularly for batting, were verified through this study. The subjects were 25 male university baseball players (age: 19.8 ± 1.0 yrs, body height: 174.5 ± 5.4 cm; body weight: 72.8 ± 5.1 kg). The participants were instructed to hit the ball placed on a tee stand. Nine types of tee-batting positions (course / height) were set for each participant depending on the upper and lower limits of the strike zone according to the baseball rules. Our main findings were as follows: 1) The swing characteristics (Depth: swing time and vertical bat angle, Height: head speed, rolling angular velocity, bat radius of rotation, horizontal bat angle, and vertical bat angle, Course: rolling angular velocity) varied with respect to ball positions, 2) Through batting skill assessment by two baseball coaches with significant experience, a good batter’s swing can be characterized as high bat speed, short swing time, and high efficiency of rotational movement around the vertical axis. Additionally, the coaches suggested that for a good batter’s swing, the vertical bat angle should be stable and smaller than 9°. These results provide useful information on assessment of bat swing training methods and exercises to hit the ball to different positions. Furthermore, this study can aid baseball coaches and/or players to objectively analyze a bat swing of a player.

Recently, research has been conducted on estimating posture and behavior using deep learning for the segmentation of object parts. For example, the segmentation of automobile parts can be used to detect a wrong-way driver or remodel an automobile. In our research, we attempt to apply deep learning to traffic censuses. During censuses, techniques to count automobiles by category from videos have been developed to save labor and improve the efficiency of work. However, these existing techniques have the problem of failing to classify automobiles with similar shapes. As a countermeasure to this problem, automobile type can be classified based on the shape of automobile parts focused on observations by surveyors. Therefore, in this research, we develop new techniques for the segmentation of automobile parts using deep learning. Furthermore, we discuss techniques for reducing the cost of re-learning by using automatically the generated training data. In the result, we obtained knowledge about the usefulness of these techniques through a demonstration experiment.

In the construction sites using crane, it is important for the crane operator to correctly grasp the sizes of the suspended load and its surrounding features, their heights and positional relationship. For supporting those works, the authors have developed the system to visualize the sizes and heights of the suspended load and its surrounding features on an image from over the suspended load, by installing a camera, a laser scanner and an inertial measurement unit on the tip of a jib. However, this system had three problems. Firstly, it was impossible to correctly recognize part of the features and ground surface when there were multiple features. Secondly, it was impossible to judge part of the slope as the ground surface when there was a slope on the ground surface within the site. Finally, it was impossible to recognize the features and the ground surface located outside the range of the crane operation. In this study, in order to solve these problems, the system is redeveloped for assisting the operator using a camera, a laser scanner and an inertial measurement unit installed on the tip of a jib, and global navigation satellite systems newly installed on the tip of a jib and the rotary center of a crane. And then, this system can generate a three-dimensional map of the operating range of the crane and display the heights and sizes of features around the crane. Finally, we evaluate in terms of usefulness and reliability through experiments.

With the spread of CGM (Consumer Generated Media), a huge amount of digital data has been accumulated on the Internet. These data are utilized for improving social sensing technologies to measure not only social and economic trends but also various kinds of phenomena such as large-scale disasters. Using habitual behavior of users, authors proposed a new social sensing method for extracting phenomena in the actual world from the difference in the habitual behavior, and proved its usefulness. To practically evaluate the versatility of the application examples of the technology, it is necessary to clarify whether the data with different user attributes are also applicable. In this study, the habitual behavior of the users is analyzed attribute by attribute, and abnormal behavior is extracted based on different behavior from the normal time for each user attribute. Demonstration experiments were conducted to verify whether it is possible to find out social trends in the actual world or social phenomena for each user attribute on a detailed granularity.

Japan Sports Agency aims at supporting distinguished performance of national members of Japan from a scientific aspect in the prioritized policy concerning improvement in international athletic ability. Focusing on the field sports, we have been developing a system for visualizing athletes’ plays using the GNSS sensor. In particular, we have been performing a research on matchup analysis of pass plays with a focus on American football. The aim of the research was to decide whether a pass is completed or incompleted by deep learning, using trajectory images of matchup of QB, WR, and DB. However, since only the track information of the finished plays was used, it failed to obtain information perceived during the play, for example, prediction of the completed pass probability during the play, or information that enables directing timely choice or modification of action from the side line during the game based on the prediction values. In this research, we attempt to predict completion probability during plays by using track information that takes into account the time term. In experiments, we made analysis of quick passes, short passes, and long passes to demonstrate its usefulness.

In our country, policies regarding sports are actively advanced towards the Tokyo 2020 Olympic Games. One of those policies, ”Sports x ICT” considers effective methods of utilizing ICT (Information and Communication Technology), such as development of measurement instruments, measurement and visualization of data, and proposals for new services in the field of sports. Against this backdrop, we have been developing the visualization system for American football games using terminal devices included GNSS and acceleration sensor. Using that system, we confirmed to grasp the effective information that are the individual condition and the motion analysis of American football players against not only players but also the college football leaders. But, in our existing research, we could not realize the strategy analysis of games that are to select the play calls depending on circumstances, to make a prediction of successful ratio, and so on. Then, in this research, we apply deep learning to the matchup analysis of pass play included offence and defense players, and then verify whether it is possible to infer the success or failure of a play from it.

Studies on tracking athletes by image processing are being made for the purpose of utilizing data in sports using ICT. However, it is impossible to correctly obtain the position information of athletes in the types of games such as American Football in which occlusions occur frequently due to the athletes’ close formation on the field. Therefore, it is a problem that the moving track of an athletes is divided, which makes it difficult to make accurate tracking. In this research, we propose a tracking method for an athlete who is robust against occlusion by using the moving tracks obtained from respective video images shot from multiple points to make them complement each other. Using this, we aim at tracking in regular games to obtain information including that about the opponents that cannot be obtained from the device with GNSS sensor.