LIP
Language and Image ProcessingSummary
LIP is an extension to the AMIS project, a national research initiative to broaden and deepen the understanding of methods for indexing and searching multimedia databases. AMIS covers the complete range of compu-tational aspects needed for multimedia retrieval from quality of service operating system delivery of multimedia items, to database technology specifically for multimedia, to data structures to capture the data and finally to analysis tools to index multi media information. As a test bed the project focuses on pictorial data, aiming to store, view and index a million pictures. Pictures were selected as they are hardest to handle both in terms of data size as well as difficulty to grasp the contents.LIP aims to enhance the multimedia aspect of the Amis project by including natural language processing capabilities. This is a challenging task both computationally as well as academically. Text about a picture bears the intention of the maker, whether it concerns the medical, arts or the cinematographic domain. Its aim is to gear the pictorial analysis to those components in the image which can be interpreted by computer vision. On the other hand, text about a picture may reveal the narrative structure of the picture. In this way it gives a compu-tational definition of the notion of pictorial intention.
LIP fits seamlessly in the existing AMIS project by combining natural language understanding with databases engineering research. The data structures resulting from natural language interpretation are different from the spatial data structures to index pictures committed in the AMIS proposal, but at the level of lingual or pictorial entity detectors their structure is akin. The combination of text steered pictorial interpretation will increase the linguistics insight in the subset of natural language elements which denote spatial representations (e.g. circle for ring, rectangle for book, above for on top of, divergence for distant, highlights for shiny, etc) and deictic expressions such hier and deze. It will also enhance and deepen the understanding of multimedia database design and methods and data structures for indexing multimedia data.
Project Duration and Funding
LIP is an NWO/SION project.LIP started in 1998.
Project Partners
The project participants are:Related Projects
Contact Information
prof. dr. F.M.G. de JongDepartment of Computer Science/CTIT, University of Twente
Postbus 217
7500 AE ENSCHEDE
Tel.: +31 53 4894193
Last modified 3 September 1998 by Thijs Westerveld