Multi-Modal AI-Driven Systems and a New Software Container

Paradigm shifts in technology always lead to new consumer interfaces. In 2019, while working as a Software Engineer at Apple, I saw an AI model integrated into an existing production software system; I knew then the game had changed. Given the advances in AI, I started thinking about the future and what software might look like. Long story short, I came across this quote while studying AI at USD a year later.

First, teams will gradually become composed of humans and non-humans working together..."

I spent a weekend brainstorming and imagining the future of work. Ideas of the Artificial Workforce and the future symbiotic relationship with the Human workforce are starting to take shape. What would a platform that allows businesses to hire non-humans look like? Thinking that in the future, companies will grow and train Artificial Capital like how they grow and train their Human Capital. The idea was that we could deliver AI and Software as the first iteration of Artificial Employees.

Generative AI, LLMs, and Automation advancements have validated this initial idea. Recently I listened to Sam Altman talk about where we are going with AI, and he spoke about two key points. The LLM war is over; we are now focused on making AI actionable. Second, in the future, there will be a new software container. Here are some further thoughts on the topic. Let's start with the concepts of a multi-modal AI-driven system.

Multi-modal AI-driven system

What is a multi-modal AI-driven system? It is a system that can leverage different types of data (such as text, images, audio, video, etc.) and various types of AI models (such as natural language processing, computer vision, speech recognition, etc.) to create a rich and interactive user experience. For example, a multi-modal AI-driven system could allow you to search for information using voice commands, see relevant images and videos on your screen, and get personalized recommendations based on your preferences and behavior.

A software container

Why do we need a new software container? A software container is a standard unit of software that packages up code and all its dependencies so that it can run reliably across different computing environments. Software containers have been widely used for deploying cloud-based applications and microservices. However, as AI becomes more prevalent and complex, we need a new software container that can handle the challenges of AI development and deployment. Some of these challenges include:

• Managing large and diverse datasets
• Ensuring data quality and security
• Training and tuning AI models
• Scaling and distributing AI models
• Monitoring and debugging AI models
• Updating and maintaining AI models

A new software container should be able to address these challenges by providing features such as:

• Data ingestion and transformation
• Data labeling and annotation
• Data governance and compliance
• Model development and testing
• Model deployment and orchestration
• Model performance and feedback
• Model lifecycle management

A new software container should also support multi-modal AI-driven systems by enabling seamless integration of different types of data and models. For example, a new software container could allow you to:

• Combine text, images, audio, video, etc., in a single data pipeline
• Use different AI models for different data modalities (e.g., NLP for text, CV for images, etc.)
• Fuse multiple AI models to create a unified output (e.g., captioning an image with text)
• Adapt to different user contexts and preferences (e.g., switching between voice and text input/output)

In conclusion, multi-modal AI-driven systems are the future of software development and user interaction. They require a new software container that can handle the complexity and diversity of AI development and deployment. By creating such a container, we can unleash the full potential of AI and augment human intelligence in unprecedented ways.

Sources:

[1] https://www.hbr.org/2020/01/ai-should-augment-human-intelligence-not-replace-it [2] https://www.usd.edu/news/2021/usd-to-offer-new-artificial-intelligence-degree-program [3] https://www.youtube.com/watch?v=8nT9jRdnXHY [4] https://www.docker.com/resources/what-container [5] https://www.forbes.com/sites/bernardmarr/2020/09/21/the-7-biggest-technology-trends-in-2020-everyone-must-get-ready-for-now/?sh=6aee8c2f6c06 [6] https://arxiv.org/pdf/2106.06069.pdf