Worldwide Sites

You have been detected as being from . Where applicable, you can see country-specific product information, offers, and pricing.

Change country/language X

  • United States

    We have redirected you to an equivalent page on your local site where you can see local pricing and promotions and purchase online.

    Stay on our U.S. site

Keyboard ALT + g to toggle grid overlay

Video Lens: Rapid Playback and Exploration of Large Video Collections and Associated Metadata

ACM Symposium on User Interface Software & Technology
2014

Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat.

[Name], Title, Company


Baseball Video Lens

Conceptually, the Video Lens framework is general enough to accommodate any large set of videos with metadata. In practice however, an implementation for any specific usage domain may impose unique design challenges.

To test out the framework for a specific domain, we created the Baseball Video Lens system using a collection of 29 Major League Baseball games with approximately 8,000 pitches in total. Each pitch has a collection of associated metadata from the PITCHf/x system.

The purpose of the system is to allow anyone to explore and search through a large collection of baseball game videos and find relevant or interesting clips based on dynamic querying of the associated metadata.

Interface Elements

Fig 1. Baseball Video Lens user interface components.

The Baseball Video Lens interface consists of four main components: three data viewing/filtering components, and the video playback window (Fig 1). More than 40 attributes are individually mapped for each pitch in the Single Attribute Controllers including ranging from basic information such as pitch type, pitch speed, batter name, and pitcher team, to more advanced metrics such as break length, spin rate, and pitch release location.


Single Attribute Controllers

Fig 2. Dynamic highlighting of events in multiple interface components.

Throughout the interface, a consistent mapping of "1 dot = 1 pitch" is followed, that is, every dot in the interface represents a single pitch in the dataset. The UI elements in each data viewing/filtering component are tightly linked to each other and support "real-time brushing" so that hovering the cursor over one component highlights the corresponding dots/events in each of other components (Fig 2). This real time "linking and brushing" interaction can hopefully expose interactions between the attributes that you weren't even looking for, and encourage further exploration of the data.

Fig 3. Selecting the slow two-seam fastballs.

On startup, the Video Player component is cycling through all pitches in the dataset. There are two ways to filter the collection of pitches. The first is to select a value, or range of values from the Single Attribute Controllers. In the example in Figure 3, we first click on 'FT' to select the two-seam fastballs, and the other controllers immediately update to show the selection. Next, we drag over a range in the "speed" controller to select the pitches between 89 and 83 mph.


Multi-Attribute Grid

The second way to select a set of pitches is by using the Multi-Attribute Grid. Initially, the Multi-Attribute Grid is set up displaying the Horizontal and Vertical location of the pitch when it crossed the plate. A lasso selection tool allows for selecting a set of pitches within the view, and in Figure 4 we are selecting a set of pitches below the strikezone.

Fig 4. Selecting pitches that near the bottom of the strikezone.

The real power of the Multi-Attribute Grid however is that you can drag any of the pitch attributes up to the grid to see how they interact. In Figure 5 for example, we drag the "speed" parameter to the color control, so the pitches are color coded based on speed. Then we drag the "type" and "speed" attributes to the grid axes, so we can compare how the speeds of the different pitch types compare to each other. Finally we select the slow two-seam fastballs for viewing.

Fig 5. Changing the variables displayed in the multi-attribute grid, and showing a different way to select the slow two-seam fastballs.

Any of the 40+ variables can be placed on either axis or the color component of the Multi-Attribute Grid, and any of the continuous variables can be mapped to the point size. This allows for thousands of different visualizations of the data to be created with very little effort (Fig 6).


Fig 6. Various views of the data in the Multi-Attribute Grid.


Example Use Cases

The following video shows the process and results for three example tasks:

  • Finding the lowest pitches hit for a home run.
  • Viewing the swings at pitches furthest from the middle of the plate.
  • Watching all double plays hit to the right side of the infield.

We think the Baseball Video Lens system could be useful for a number of different groups, including serious baseball fans, baseball teams/scouting departments, broadcast organizations, and baseball journalists. While each of these user groups might have different goals, they could all use the ability to find collections of clips within a large set of baseball videos.


More Information

For more information, please see the research paper or watch the longer research video.

If you have any questions please contact Justin Matejka through email Justin.Matejka@Autodesk.com or Twitter @JustinMatejka.

Section Title

Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat. Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat. Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat. Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat.

Fig 1. Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua.

Section Title

Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat. Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua.

Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat. Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua.

Fig 2. Lorem ipsum dolor sit amet, consectetur adipiscing elit.

Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat. Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua.

Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat. Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua.

Section Title

  • List Item 1
  • List Item 2
  • List Item 3
  • List Item 4
  • List Item 5
  • List Item 6
  • List Item 7

Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat. Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua.

Abstract

We present Video Lens, a framework which allows users to visualize and interactively explore large collections of videos and associated metadata. The primary goal of the framework is to let users quickly find relevant sections within the videos and play them back in rapid succession. The individual UI elements are linked and highly interactive, supporting a faceted search paradigm and encouraging exploration of the data set. We demonstrate the capabilities and specific scenarios of Video Lens within the domain of professional baseball videos. A user study with 12 participants indicates that Video Lens efficiently supports a diverse range of powerful yet desirable video query tasks, while a series of interviews with professionals in the field demonstrates the framework’s benefits and future potential.

File downloads

  • Additional files

    Datasets and sample images. Python source code.

  • Media kit files

    Paper, Video, Images and Datasets

Related Publications

Related Projects

Visualization & Visual Analytics

Visual data representations leverage the power of human perception to process complex information, and through interaction, garner new insights. Our research focuses on visualizing data from a wide variety of domains and fundamentally tackles the question, what makes a visualization effective? We explore novel visual encodings and interaction techniques, multiscale approaches, and even simulation to bridge human and automated analysis of multivariate, time-series, and graph data, ultimately aiding in hypothesis generation, testing, and sense making.

Heading

Descriptive text. Lorem ipsum dolor sit amet, consectetur adipisicing elit, sed do eiusmod tempor incididunt magna aliqua in reprehenderit.

Heading

Descriptive text. Lorem ipsum dolor sit amet, consectetur adipisicing elit, sed do eiusmod tempor incididunt magna aliqua in reprehenderit.