Multimodality

We will almost certainly need to use a custom editor component for multimodal projects.

Custom editor learningsarrow-up-right

Audio

See SIL Transcriberarrow-up-right app

Audio editing pluginarrow-up-right (though double check license, and it’s not super functional at this point.)

Screenshot 2023-12-20 at 8.08.17 AM.png

We should explore how we might integrate AVTT software into VS Code as rendered components.

Image

Klappy has made a simple POC plugin generating images using scripture prompts.

I suspect there are a number of image editing plugins. We could use these to mark up images before processing with a multimodal LLM.

Cf. Custom editor sample extension for image editingarrow-up-right

Last updated