Multimodality

We will almost certainly need to use a custom editor component for multimodal projects.

Custom editor learnings

Audio

See SIL Transcriber app

Audio editing plugin (though double check license, and it’s not super functional at this point.)

We should explore how we might integrate AVTT software into VS Code as rendered components.

Image

Klappy has made a simple POC plugin generating images using scripture prompts.

I suspect there are a number of image editing plugins. We could use these to mark up images before processing with a multimodal LLM.

Cf. Custom editor sample extension for image editing

Last updated