Multimedia IR systems

My first step in the role of an SME was getting to know what Multimedia Data is.
I started learning it as I understood that my whole photo album is Multimedia Data itself.

Then I realised that I need to use a solution to search for Multimedia Data - Multimedia IR systems.

However, there are many Multimedia IR systems, and it was a long way to select one of them to start searching more efficiently.
Thus, several Multimedia IR systems, that are present on the market and available for end users, were analyzed.

I described the process of analysis and comparison for Multimedia IR systems here.
Consequently, I was able to choose the most suitable system for me.

Metadata

A short usage of these systems gave me an understanding that Multimedia IR systems significantly rely on the metadata as a main information source that is used for a search following different criteria, like device, time, location, etc.

Therefore, the presence and consistency of metadata are very important for Multimedia IR systems to operate, as well as for other purposes, please see Three Cheers for Embedded Metadata for more details.

Working experience with one of those systems shows that search results might be incomplete or even incorrect due to inconsistent or not-existing metadata.
Metadata is usually kept inside each file as a header with a predefined format.
There are plenty of metadata formats like Exif, IPTC or XMP.

The reasons for incorrect and absent metadata are quite different - from not setting proper data on the device that takes photos to privacy issues.

This resource describes metadata importance in more detail and there is information about other reasons for incorrect and absent metadata.

Examples of incorrect search results

Let's consider two examples of incorrect search results.
In the first case, we tried to search for photos with existing Geo positions and to see them on a map.
A search returned four photos taken near Ghana, marked by a red circle.

In fact, nobody took photos near Ghana.

In the second case, we searched for photos taken in 2011.
We got one video that was recognized as March 2020, underlined in red.

The system has provided such results due to incorrect metadata.

Multimedia Organizers

That being the case, I understood that Multimedia IR systems are not so effective without other tools, often called Multimedia Organizers.

Usually, Multimedia Organizers provide automation for importing a new photo set into existing photo albums and it is not an implemented feature in Multimedia IR systems.
In addition, they might identify and address some issues that shall be fixed before a user starts searching data using the Multimedia IR systems.

Integration

Then I started studying Multimedia IR systems the FVA Tool Set shall integrate with.
Integration with offline Multimedia IR systems is implemented through integration with digiKam, and integration with online Multimedia IR systems is implemented through integration with Google Photos.

As Multimedia IR systems perform a search by certain criteria, based on multimedia metadata, the FVA Software shall be able to work with the same multimedia metadata to be integrated with Multimedia IR systems.

Integration with digiKam

For digiKam, I found that the Collections setting shall be used to define how it is configured to search through multimedia data.
Monitor the albums for external changes (requires restart) setting shall be enabled.
Taking into account this setting, digiKam will pick up external changes in the collection as soon as FVA Tool Set puts it in.

However, from the FVA Tool Set side, it is required to provide a possibility to configure storage in a way to be able to put multimedia data processed and to be integrated with digiKam.
Therefore, a separate UI screen shall be created to ask a user what storage location, digiKam should be used for incoming multimedia data.

For example, a directory C:\FVA is selected for Local Collections as shown in the picture above.
Therefore, when a user wants to integrate multimedia data after processing by FVA Tool Set, he/she shall be able to select the C:\FVA directory as a directory in UI to put the processed multimedia data in.

As soon as it is done, FVA Tool Set will put multimedia data into a selected directory and the Multimedia IR system will be able to search through it.

Integration with Google Photos

Integration with Google Photos is implemented in the same way.
First of all, it's worth noting that Google Photos shall be configured to be integrated with FVA Tool Set.
Thus, appropriate file types and folders shall be set up as in the figure below:

For example, Pictures and fromDevices directories are selected for the folder settings as shown above.
It means that when a user wants to integrate multimedia data after processing by FVA Tool Set, he/she shall be able to select the fromDevices or Pictures directory on the UI screen, as a directory to put processed multimedia data in.

As soon as it is done, FVA Tool Set will put multimedia data into a selected directory and Google Photos will be able to search through it as soon as it synchronizes multimedia data.