# Multimodality

We will almost certainly need to use a custom editor component for multimodal projects.

### Links/Resources

[Custom editor learnings](https://timheuer.com/blog/resx-editor-for-visual-studio-code/)

#### Audio

See [SIL Transcriber](https://software.sil.org/siltranscriber/) app

[Audio editing plugin](https://marketplace.visualstudio.com/items?itemName=chocolate-pie.sound-editor\&ssr=false#overview) (though double check license, and it’s not super functional at this point.)

![Screenshot 2023-12-20 at 8.08.17 AM.png](https://4219285846-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FCrkjWLsYWix5ZOP9QKP1%2Fuploads%2FU8lNZV0rIqjXSdvOOHsl%2FScreenshot%202023-12-20%20at%208.08.17%E2%80%AFAM.png?alt=media\&token=9314e256-447a-48ad-b7bf-21222e6ff8a8)

We should explore how we might integrate AVTT software into VS Code as rendered components. 

#### Image

Klappy has made a simple POC plugin generating images using scripture prompts. 

I suspect there are a number of image editing plugins. We could use these to mark up images before processing with a multimodal LLM.

Cf. [Custom editor sample extension for image editing](https://github.com/microsoft/vscode-extension-samples/blob/main/custom-editor-sample/README.md)