In this paper, we propose a framework for a system that dynamically selects and plays multimedia files from a large data repository. The performance is generated based on the technical, semantic and relational textual annotation of the data as well as context-sensitive rules and patterns of selection discovered with the aid of the system during the performance preparation phase. We borrow concepts from the fields of discourse analysis [2] and rhetorical structure [4] as the theoretical basis for our work. To validate the framework, researchers from the Computer Science department are developing a Flash prototype with data created and annotated by a research group from the department of Design Art.