Multimodality

We will almost certainly need to use a custom editor component for multimodal projects.

Custom editor learnings

Audio

See SIL Transcriber app

Audio editing plugin (though double check license, and itโ€™s not super functional at this point.)

Screenshot 2023-12-20 at 8.08.17โ€ฏAM.png

We should explore how we might integrate AVTT software into VS Code as rendered components.

Image

Klappy has made a simple POC plugin generating images using scripture prompts.

I suspect there are a number of image editing plugins. We could use these to mark up images before processing with a multimodal LLM.

Cf. Custom editor sample extension for image editing

Last updated