📌Overview

Purpose: The Unreal Engine Generative AI Support Plugin aims to simplify the integration of various generative AI models into game development, allowing artists and developers to focus on creative tasks without the complexity of LLM/GenAI integration.

Overview: This plugin is designed for Unreal Engine users, providing compatibility with several state-of-the-art generative AI models such as OpenAI's GPT-4o and Claude Sonnet 3.7. It focuses on creating a user-friendly experience for implementing AI features in games and interactive experiences, all while maintaining a long-term support framework.

Key Features:

• Model Integration: Supports multiple LLM/GenAI models for diverse applications in game development, including APIs for OpenAI, Deepseek, and more, targeting real-time interactions and creative generation.

• MCP Support: Integrates a Model Control Protocol (MCP) for enhanced control over scene objects in Unreal Engine, enabling tasks like spawning and manipulating entities directly through prompts or scripts.

• Blueprint Generation: Automates the creation of Blueprints and functions, streamlining the workflow for developers by providing an easy mechanism to generate complex behaviors and interactions in Unreal projects.

---

Unreal Engine Generative AI Support Plugin

Overview

The Unreal Engine Generative AI Support Plugin simplifies game development by managing the integration of various LLM/GenAI models. It currently supports multiple APIs including OpenAI's GPT-4o and Claude Sonnet, and focuses exclusively on game development relevant APIs. The plugin is designed for Unreal Engine 5.1 or higher.

Warning: This plugin is under rapid development. Use with caution in non-production environments and ensure proper version control.

Current Progress

LLM/GenAI API Support

• OpenAI API Support:

  • Chat API:
    • gpt-4o, gpt-4o-mini: ✅
    • gpt-4.5-preview: 🛠️
    • Additional models: 🚧
  • DALL-E and Vision API: ❌
  • Realtime API: 🛠️

• Anthropic Claude API Support:

  • Chat API:
    • Various models: ✅
  • Vision API: 🚧

• Other API Supports:

  • XAI, Google Gemini, Meta AI, and Deepseek APIs with varying levels of support.

Table of Contents

• Setting API Keys
• Setting up MCP
• Adding the plugin to your project
• Fetching the Latest Plugin Changes
• Usage
• Known Issues
• Contribution Guidelines

Setting API Keys

Set the environment variable PS_<ORGNAME> to your API key.

For Windows:

setx PS_<ORGNAME> "your api key"

For Linux/MacOS:

echo "export PS_<ORGNAME>='yourkey'" >> ~/.zshrc
source ~/.zshrc

Where <ORGNAME> can be: PS_OPENAIAPIKEY, PS_DEEPSEEKAPIKEY, etc.

Setting up MCP

Note: Skip this section if your project only uses LLM APIs.

1. Install a client, such as Claude Desktop App or Cursor IDE.
2. Setup the MCP config in the appropriate JSON file.
3. Install MCP CLI with pip install mcp[cli].
4. Enable the Python plugin in Unreal Engine.

Adding the Plugin to Your Project

With Git

1. Add the Plugin Repository as a Submodule:

   git submodule add https://github.com/prajwalshettydev/UnrealGenAISupport Plugins/GenerativeAISupport
   

2. Enable the Plugin in Unreal Editor.

With Perforce

(TBD)

With Unreal Marketplace

Coming soon.

Fetching the Latest Plugin Changes

Use the following command to pull changes:

cd Plugins/GenerativeAISupport
git pull origin main

Usage

The plugin supports various APIs for Chat and Structured Outputs, primarily from OpenAI and DeepSeek.

OpenAI

Example code for using Chat in C++:

// Sample function to use OpenAI Chat API
void SomeDebugSubsystem::CallGPT(const FString& Prompt, 
    const TFunction<void(const FString&, const FString&, bool)>& Callback) {
    // Implementation 
}

DeepSeek API

Example code for reasoning:

// Sample function to use DeepSeek API
FGenDSeekChatSettings ReasoningSettings;
// Implementation 

Known Issues

• Nodes fail to connect properly with MCP.
• Limited support for complex material generation.
• Issues with LLM-generated Python scripts.

Contribution Guidelines

Setting up for Development

Install the required Python package:

pip install unreal

Project Structure

More details will be added soon.

References

• OpenAI API Documentation: OpenAI
• Deepseek API Documentation: Deepseek
• Model Control Protocol Documentation: MCP