Connect with us


Apple’s new AI model can edit images using natural language

Apple has made MGIE available on GitHub and also released a web demo on Hugging Face Spaces.

Apple researchers have released a new AI model that allows users to describe in plain language what they want to change in a photo without the need for photo editing software.

The MGIE model, which Apple worked on with the University of California, Santa Barbara, allows users to crop, resize, flip and add filters to images using textual prompts.

MGIE, which stands for MLLM-Guided Image Editing, can be used for both simple and more complex image editing tasks, such as changing certain objects in a photo to give them a different shape or make them more vivid. The model combines two different applications of multimodal language models. First, it learns to interpret the user’s cues. Then it “imagines” what the edit will look like (for example, a request to make the sky in a photo bluer becomes an increase in brightness on the sky portion of the image).

When editing a photo using MGIE, the user just needs to type what they want to change in the image. The article gives an example of editing an image of a pepperoni pizza. By typing the query “make it healthier”, vegetable toppings can be added. A photo of tigers in the Sahara looks dark, but after telling the model to “add more contrast to simulate more light”, the image becomes brighter.

Apple has made MGIE available on GitHub and also released a web demo on Hugging Face Spaces. The company has not said what plans it has for the model beyond research.

Some image generation platforms, such as OpenAI’s DALL-E 3, can perform simple editing tasks on photos they create with text input. Photoshop creator Adobe, the company most people turn to for image editing, also has its own artificial intelligence editing model. Its Firefly artificial intelligence model provides generative fill that adds generated backgrounds to photos.

Apple hasn’t been a major player in generative AI, unlike Microsoft, Meta or Google, but Apple CEO Tim Cook has said the company wants to add more AI features to its devices this year. Apple researchers had already released an open-source machine learning framework called MLX in December to make it easier to train AI models on Apple chips.