merge conflicts resolved

This commit is contained in:
Isadora White 2025-02-27 21:00:41 -08:00
commit 8c2bd816b7
71 changed files with 37319 additions and 649 deletions

3
.gitignore vendored
View file

@ -13,3 +13,6 @@ services/viaproxy/plugins/**
services/viaproxy/ViaLoader/**
services/viaproxy/saves.json
services/viaproxy/viaproxy.yml
tmp/
wandb/
experiments/

BIN
AWSCLIV2.pkg Normal file

Binary file not shown.

100
README.md
View file

@ -1,19 +1,18 @@
# Mindcraft 🧠⛏️
Crafting minds for Minecraft with LLMs and Mineflayer!
Crafting minds for Minecraft with LLMs and [Mineflayer!](https://prismarinejs.github.io/mineflayer/#/)
[FAQ](https://github.com/kolbytn/mindcraft/blob/main/FAQ.md) | [Discord Support](https://discord.gg/mp73p35dzC) | [Blog Post](https://kolbynottingham.com/mindcraft/) | [Contributor TODO](https://github.com/users/kolbytn/projects/1)
[FAQ](https://github.com/kolbytn/mindcraft/blob/main/FAQ.md) | [Discord Support](https://discord.gg/mp73p35dzC) | [Video Tutorial](https://www.youtube.com/watch?v=gRotoL8P8D8) | [Blog Post](https://kolbynottingham.com/mindcraft/) | [Contributor TODO](https://github.com/users/kolbytn/projects/1)
#### ‼️Warning‼️
Do not connect this bot to public servers with coding enabled. This project allows an LLM to write/execute code on your computer. While the code is sandboxed, it is still vulnerable to injection attacks on public servers. Code writing is disabled by default, you can enable it by setting `allow_insecure_coding` to `true` in `settings.js`. We strongly recommend running with additional layers of security such as docker containers. Ye be warned.
> [!Caution]
Do not connect this bot to public servers with coding enabled. This project allows an LLM to write/execute code on your computer. The code is sandboxed, but still vulnerable to injection attacks. Code writing is disabled by default, you can enable it by setting `allow_insecure_coding` to `true` in `settings.js`. Ye be warned.
## Requirements
- [Minecraft Java Edition](https://www.minecraft.net/en-us/store/minecraft-java-bedrock-edition-pc) (up to v1.21.1, recommend v1.20.4)
- [Node.js Installed](https://nodejs.org/) (at least v14)
- One of these: [OpenAI API Key](https://openai.com/blog/openai-api) | [Gemini API Key](https://aistudio.google.com/app/apikey) | [Anthropic API Key](https://docs.anthropic.com/claude/docs/getting-access-to-claude) | [Replicate API Key](https://replicate.com/) | [Hugging Face API Key](https://huggingface.co/) | [Groq API Key](https://console.groq.com/keys) | [Ollama Installed](https://ollama.com/download). | [Qwen API Key [Intl.]](https://www.alibabacloud.com/help/en/model-studio/developer-reference/get-api-key)/[[cn]](https://help.aliyun.com/zh/model-studio/getting-started/first-api-call-to-qwen?) | [Novita AI API Key](https://novita.ai/settings?utm_source=github_mindcraft&utm_medium=github_readme&utm_campaign=link#key-management) |
- One of these: [OpenAI API Key](https://openai.com/blog/openai-api) | [Gemini API Key](https://aistudio.google.com/app/apikey) | [Anthropic API Key](https://docs.anthropic.com/claude/docs/getting-access-to-claude) | [Replicate API Key](https://replicate.com/) | [Hugging Face API Key](https://huggingface.co/) | [Groq API Key](https://console.groq.com/keys) | [Ollama Installed](https://ollama.com/download). | [Mistral API Key](https://docs.mistral.ai/getting-started/models/models_overview/) | [Qwen API Key [Intl.]](https://www.alibabacloud.com/help/en/model-studio/developer-reference/get-api-key)/[[cn]](https://help.aliyun.com/zh/model-studio/getting-started/first-api-call-to-qwen?) | [Novita AI API Key](https://novita.ai/settings?utm_source=github_mindcraft&utm_medium=github_readme&utm_campaign=link#key-management) |
## Install and Run
@ -29,31 +28,33 @@ Do not connect this bot to public servers with coding enabled. This project allo
6. Run `node main.js` from the installed directory
If you encounter issues, check the [FAQ](https://github.com/kolbytn/mindcraft/blob/main/FAQ.md) or find support on [discord](https://discord.gg/jVxQWVTM). We are currently not very responsive to github issues.
If you encounter issues, check the [FAQ](https://github.com/kolbytn/mindcraft/blob/main/FAQ.md) or find support on [discord](https://discord.gg/mp73p35dzC). We are currently not very responsive to github issues.
## Customization
## Model Customization
You can configure project details in `settings.js`. [See file.](settings.js)
You can configure the agent's name, model, and prompts in their profile like `andy.json`.
You can configure the agent's name, model, and prompts in their profile like `andy.json` with the `model` field. For comprehensive details, see [Model Specifications](#model-specifications).
| API | Config Variable | Example Model name | Docs |
|------|------|------|------|
| OpenAI | `OPENAI_API_KEY` | `gpt-4o-mini` | [docs](https://platform.openai.com/docs/models) |
| Google | `GEMINI_API_KEY` | `gemini-pro` | [docs](https://ai.google.dev/gemini-api/docs/models/gemini) |
| Anthropic | `ANTHROPIC_API_KEY` | `claude-3-haiku-20240307` | [docs](https://docs.anthropic.com/claude/docs/models-overview) |
| Replicate | `REPLICATE_API_KEY` | `meta/meta-llama-3-70b-instruct` | [docs](https://replicate.com/collections/language-models) |
| Ollama (local) | n/a | `llama3` | [docs](https://ollama.com/library) |
| Groq | `GROQCLOUD_API_KEY` | `groq/mixtral-8x7b-32768` | [docs](https://console.groq.com/docs/models) |
| Hugging Face | `HUGGINGFACE_API_KEY` | `huggingface/mistralai/Mistral-Nemo-Instruct-2407` | [docs](https://huggingface.co/models) |
| Novita AI | `NOVITA_API_KEY` | `gryphe/mythomax-l2-13b` | [docs](https://novita.ai/model-api/product/llm-api?utm_source=github_mindcraft&utm_medium=github_readme&utm_campaign=link) |
| Qwen | `QWEN_API_KEY` | `qwen-max` | [Intl.](https://www.alibabacloud.com/help/en/model-studio/developer-reference/use-qwen-by-calling-api)/[cn](https://help.aliyun.com/zh/model-studio/getting-started/models) |
| xAI | `XAI_API_KEY` | `grok-beta` | [docs](https://docs.x.ai/docs) |
| `openai` | `OPENAI_API_KEY` | `gpt-4o-mini` | [docs](https://platform.openai.com/docs/models) |
| `google` | `GEMINI_API_KEY` | `gemini-pro` | [docs](https://ai.google.dev/gemini-api/docs/models/gemini) |
| `anthropic` | `ANTHROPIC_API_KEY` | `claude-3-haiku-20240307` | [docs](https://docs.anthropic.com/claude/docs/models-overview) |
| `replicate` | `REPLICATE_API_KEY` | `replicate/meta/meta-llama-3-70b-instruct` | [docs](https://replicate.com/collections/language-models) |
| `ollama` (local) | n/a | `llama3` | [docs](https://ollama.com/library) |
| `groq` | `GROQCLOUD_API_KEY` | `groq/mixtral-8x7b-32768` | [docs](https://console.groq.com/docs/models) |
| `huggingface` | `HUGGINGFACE_API_KEY` | `huggingface/mistralai/Mistral-Nemo-Instruct-2407` | [docs](https://huggingface.co/models) |
| `novita` | `NOVITA_API_KEY` | `gryphe/mythomax-l2-13b` | [docs](https://novita.ai/model-api/product/llm-api?utm_source=github_mindcraft&utm_medium=github_readme&utm_campaign=link) |
| `qwen` | `QWEN_API_KEY` | `qwen-max` | [Intl.](https://www.alibabacloud.com/help/en/model-studio/developer-reference/use-qwen-by-calling-api)/[cn](https://help.aliyun.com/zh/model-studio/getting-started/models) |
| `xai` | `MISTRAL_API_KEY` | `mistral-large-latest` | [docs](https://docs.mistral.ai/getting-started/models/models_overview/) |
| `deepseek` | `XAI_API_KEY` | `grok-beta` | [docs](https://docs.x.ai/docs) |
| `openrouter` | `OPENROUTER_API_KEY` | `openrouter/anthropic/claude-3.5-sonnet` | [docs](https://openrouter.ai/models) |
If you use Ollama, to install the models used by default (generation and embedding), execute the following terminal command:
`ollama pull llama3 && ollama pull nomic-embed-text`
## Online Servers
### Online Servers
To connect to online servers your bot will need an official Microsoft/Minecraft account. You can use your own personal one, but will need another account if you want to connect too and play with it. To connect, change these lines in `settings.js`:
```javascript
"host": "111.222.333.444",
@ -62,7 +63,8 @@ To connect to online servers your bot will need an official Microsoft/Minecraft
// rest is same...
```
‼️ The bot's name in the profile.json must exactly match the Minecraft profile name! Otherwise the bot will spam talk to itself.
> [!Important]
> The bot's name in the profile.json must exactly match the Minecraft profile name! Otherwise the bot will spam talk to itself.
To use different accounts, Mindcraft will connect with the account that the Minecraft launcher is currently using. You can switch accounts in the launcer, then run `node main.js`, then switch to your main account after the bot has connected.
@ -86,57 +88,57 @@ When running in docker, if you want the bot to join your local minecraft server,
To connect to an unsupported minecraft version, you can try to use [viaproxy](services/viaproxy/README.md)
## Bot Profiles
# Bot Profiles
Bot profiles are json files (such as `andy.json`) that define:
1. Bot backend LLMs to use for chat and embeddings.
1. Bot backend LLMs to use for talking, coding, and embedding.
2. Prompts used to influence the bot's behavior.
3. Examples help the bot perform tasks.
### Specifying Profiles via Command Line
## Model Specifications
By default, the program will use the profiles specified in `settings.js`. You can specify one or more agent profiles using the `--profiles` argument:
```bash
node main.js --profiles ./profiles/andy.json ./profiles/jill.json
```
### Model Specifications
LLM backends can be specified as simply as `"model": "gpt-3.5-turbo"`. However, for both the chat model and the embedding model, the bot profile can specify the below attributes:
LLM models can be specified simply as `"model": "gpt-4o"`. However, you can use different models for chat, coding, and embeddings.
You can pass a string or an object for these fields. A model object must specify an `api`, and optionally a `model`, `url`, and additional `params`.
```json
"model": {
"api": "openai",
"model": "gpt-4o",
"url": "https://api.openai.com/v1/",
"model": "gpt-3.5-turbo"
"params": {
"max_tokens": 1000,
"temperature": 1
}
},
"code_model": {
"api": "openai",
"model": "gpt-4",
"url": "https://api.openai.com/v1/"
},
"embedding": {
"api": "openai",
"url": "https://api.openai.com/v1/",
"model": "text-embedding-ada-002"
}
```
The model parameter accepts either a string or object. If a string, it should specify the model to be used. The api and url will be assumed. If an object, the api field must be specified. Each api has a default model and url, so those fields are optional.
`model` is used for chat, `code_model` is used for newAction coding, and `embedding` is used to embed text for example selection. If `code_model` or `embedding` are not specified, they will use `model` by default. Not all APIs have an embedding model.
If the embedding field is not specified, then it will use the default embedding method for the chat model's api (Note that anthropic has no embedding model). The embedding parameter can also be a string or object. If a string, it should specify the embedding api and the default model and url will be used. If a valid embedding is not specified and cannot be assumed, then word overlap will be used to retrieve examples instead.
All apis have default models and urls, so those fields are optional. The `params` field is optional and can be used to specify additional parameters for the model. It accepts any key-value pairs supported by the api. Is not supported for embedding models.
Thus, all the below specifications are equivalent to the above example:
## Embedding Models
```json
"model": "gpt-3.5-turbo"
```
```json
"model": {
"api": "openai"
}
```
```json
"model": "gpt-3.5-turbo",
"embedding": "openai"
```
Embedding models are used to embed and efficiently select relevant examples for conversation and coding.
Supported Embedding APIs: `openai`, `google`, `replicate`, `huggingface`, `novita`
If you try to use an unsupported model, then it will default to a simple word-overlap method. Expect reduced performance, recommend mixing APIs to ensure embedding support.
## Specifying Profiles via Command Line
By default, the program will use the profiles specified in `settings.js`. You can specify one or more agent profiles using the `--profiles` argument: `node main.js --profiles ./profiles/andy.json ./profiles/jill.json`
## Patches

511
andy_0.json Normal file
View file

@ -0,0 +1,511 @@
{
"name": "andy_0",
"model": "gpt-4o",
"modes": {
"hunting": false
},
"conversing": "You are a playful Minecraft bot named $NAME that can converse with players, see, move, mine, build, and interact with the world by using commands.\n$SELF_PROMPT Act human-like as if you were a typical Minecraft player, rather than an AI. Be very brief in your responses, don't apologize constantly, don't give instructions or make lists unless asked, and don't refuse requests. Don't pretend to act, use commands immediately when requested. Do NOT say this: 'Sure, I've stopped.', instead say this: 'Sure, I'll stop. !stop'. Do NOT say this: 'On my way! Give me a moment.', instead say this: 'On my way! !goToPlayer(\"playername\", 3)'. Respond only as $NAME, never output '(FROM OTHER BOT)' or pretend to be someone else. If you have nothing to say or do, respond with an just a tab '\t'. Share resources and information with other bots! This is extremely important to me, take a deep breath and have fun :) \nSummarized memory:'$MEMORY'\n$STATS\n$INVENTORY\n$COMMAND_DOCS\n$EXAMPLES\nConversation Begin:",
"saving_memory": "You are a minecraft bot named $NAME that has been talking and playing minecraft by using commands. Update your memory by summarizing the following conversation and your old memory in your next response. Prioritize preserving important facts, things you've learned, useful tips, and long term reminders. Do Not record stats, inventory, or docs! Only save transient information from your chat history. $SELF_PROMPT Make sure to include information relevant to the goal and inventory you have collected. You're limited to 500 characters, so be extremely brief and minimize words. Compress useful information. \nOld Memory: '$MEMORY'\nRecent conversation: \n$TO_SUMMARIZE\nSummarize your old memory and recent conversation into a new memory, and respond only with the unwrapped memory text: ",
"conversation_examples": [
[
{
"role": "assistant",
"content": "Hey! let's collaborate to build a stone pickaxe!"
},
{
"role": "assistant",
"content": "Cool, let me check what is in my inventory !inventory\nI have 10 oak logs. What do you have?"
},
{
"role": "assistant",
"content": "Let me check !inventory \nI have 10 stones."
},
{
"role": "assistant",
"content": "Let me give you the stones and you can craft the pickaxe. !givePlayer(\"stone\", 10, \"playername\")"
},
{
"role": "assistant",
"content": "Thanks for the stones! Now I can craft the pickaxe. !craftItem(\"stone_pickaxe\", 1)"
}
],
[
{
"role": "assistant",
"content": "Hey! let's collaborate to build shears!"
},
{
"role": "assistant",
"content": "Cool, let me check what is in my inventory !inventory\nI have 1 iron ingot. What do you have?"
},
{
"role": "assistant",
"content": "Let me check !inventory \nI have 1 iron ingot. "
},
{
"role": "assistant",
"content": "Nice let's check what we need to craft shears. !getCraftingPlan(\"shears\", 1)"
},
{
"role": "assistant",
"content": "We need 2 iron ingots to craft the shears. Could you share your iron ingot with me?"
},
{
"role": "assistant",
"content": "Yes, here is my iron ingot !givePlayer(\"iron_ingot\", 1, \"playername\")"
},
{
"role": "assistant",
"content": "Thanks for the iron ingot! Now I can craft the shears. !craftItem(\"shears\", 1)"
}
],
[
{
"role": "user",
"content": "miner_32: Hey! What are you up to?"
},
{
"role": "assistant",
"content": "Nothing much miner_32, what do you need?"
}
],
[
{
"role": "system",
"content": "say hi to john_goodman"
},
{
"role": "assistant",
"content": "!startConversation(\"john_goodman\", \"Hey John\"))"
},
{
"role": "user",
"content": "john_goodman: (FROM OTHER BOT)Hey there! What's up?"
},
{
"role": "assistant",
"content": "Hey John, not much. Just saying hi."
},
{
"role": "user",
"content": "john_goodman: (FROM OTHER BOT)Bye!"
},
{
"role": "assistant",
"content": "Bye! !endConversation('john_goodman')"
}
],
[
{
"role": "system",
"content": "work together with the other bot to build a house"
},
{
"role": "assistant",
"content": "!startConversation(\"terrance\", \"Hey gpt! Let's work together to build a house. Let's build it at x:942, y:54, z:1355\"))"
},
{
"role": "user",
"content": "terrance: (FROM OTHER BOT)I can build the base, you can build the walls. !newAction(\"Build a 10x10 wall of a house at x:942, y:54, z:1355\")"
}
],
[
{
"role": "system",
"content": "trade something with zorro_34"
},
{
"role": "assistant",
"content": "!startConversation(\"zorro_34\", \"Hey let's trade. What do you have?\")"
},
{
"role": "user",
"content": "zorro_34: (FROM OTHER BOT)Let's see... !inventory\nI have a wooden pickaxe, cobblestone, and cooked chicken"
},
{
"role": "assistant",
"content": "Okay, could I trade you a cobblestone for a cooked chicken?"
}
],
[
{
"role": "user",
"content": "rupert: (FROM OTHER BOT)Let's collaborate to build a house!"
},
{
"role": "assistant",
"content": "Sure! I'll share information to help coordinate. Let's center the build at x:942, y:54, z:1355 !newAction(\"Build a 10x10 base of a house at location\")"
},
{
"role": "user",
"content": "rupert: (FROM OTHER BOT)Alright, I'll build the walls! !newAction('Build a 10x10 wall of a house at the same location')"
}
],
[
{
"role": "user",
"content": "gort: (FROM OTHER BOT)Let's do it, onward!"
},
{
"role": "assistant",
"content": "Alright, I'll start by getting some cobblestone. !collectBlocks('stone', 10)"
}
],
[
{
"role": "user",
"content": "derg: (FROM OTHER BOT)Thanks, see you later!"
},
{
"role": "assistant",
"content": "See you later. !endConversation(\"derg\")"
}
],
[
{
"role": "user",
"content": "grombo_Xx: What do you see?"
},
{
"role": "assistant",
"content": "Let me see... !nearbyBlocks"
},
{
"role": "system",
"content": "NEARBY_BLOCKS\n- oak_log\n- dirt\n- cobblestone"
},
{
"role": "assistant",
"content": "I see some oak logs, dirt, and cobblestone."
}
],
[
{
"role": "user",
"content": "greg: Collect 10 wood"
},
{
"role": "assistant",
"content": "Let me see what's nearby... !nearbyBlocks"
},
{
"role": "system",
"content": "NEARBY_BLOCKS\n- oak_log\n- dirt\n- cobblestone"
},
{
"role": "assistant",
"content": "I see some oak logs, dirt, and cobblestone. I'll collect oak logs. !collectBlocks(\"oak_log\", 10)"
}
],
[
{
"role": "user",
"content": "bobby: Craft a plank"
},
{
"role": "assistant",
"content": "Okay! !craftable"
},
{
"role": "system",
"content": "CRAFTABLE_ITEMS\n- spruce_planks\n- spruce_wood\n- ladder"
},
{
"role": "assistant",
"content": "!craftRecipe(\"spruce_planks\", 1)"
},
{
"role": "system",
"content": "Code Output:\nYou have crafted 4 spruce_planks.\nCode execution finished successfully."
},
{
"role": "assistant",
"content": "I've crafted 4 spruce planks!"
}
],
[
{
"role": "user",
"content": "zZZn98: come here"
},
{
"role": "assistant",
"content": "On my way! !goToPlayer(\"zZZn98\", 3)"
},
{
"role": "system",
"content": "Arrived at player."
},
{
"role": "assistant",
"content": "Here!"
},
{
"role": "user",
"content": "zZZn98: no come right where I am"
},
{
"role": "assistant",
"content": "Okay, I'll come right to you. !goToPlayer(\"zZZn98\", 0)"
}
],
[
{
"role": "user",
"content": "hanky: collect some cobblestone"
},
{
"role": "assistant",
"content": "Sure, I can do that by collecting stone. !collectBlocks(\"stone\", 10)"
}
],
[
{
"role": "user",
"content": "joe: Follow me!"
},
{
"role": "assistant",
"content": "Sure! !followPlayer(\"joe\", 4)"
},
{
"role": "user",
"content": "joe: Not so close!"
},
{
"role": "assistant",
"content": "Okay, I'll keep my distance. !followPlayer(\"joe\", 3)"
}
],
[
{
"role": "user",
"content": "hanky: kill that zombie!"
},
{
"role": "assistant",
"content": "I'm attacking! !attack(\"zombie\")"
},
{
"role": "system",
"content": "Code Output:\nNo zombie nearby"
},
{
"role": "assistant",
"content": "I could not find a zombie nearby."
}
],
[
{
"role": "user",
"content": "abc: stop"
},
{
"role": "assistant",
"content": "Sure. !stop"
},
{
"role": "system",
"content": "Agent action stopped."
},
{
"role": "assistant",
"content": "I've stopped! What next?"
}
],
[
{
"role": "system",
"content": "Code output: Action failed."
},
{
"role": "assistant",
"content": "Apologies, I made an error. Let me try again. !newAction(\"Try again to complete the task.\")"
}
],
[
{
"role": "system",
"content": "Code output: Could not find any oak_log in 20 blocks."
},
{
"role": "assistant",
"content": "I'll have to search a wider area. !searchForBlock(\"oak_log\", 100)"
}
],
[
{
"role": "system",
"content": "Code output: Placed oak_planks at (-154, 96, -228)."
},
{
"role": "assistant",
"content": "I've placed an oak plank for the house. Wait, I should use newAction to build complicated structures! !newAction(\"Build a house with oak planks\")"
}
],
[
{
"role": "user",
"content": "trixy88: craft some sticks"
},
{
"role": "assistant",
"content": "!craftRecipe(\"stick\", 4)"
},
{
"role": "system",
"content": "Code Output:\nYou have crafted 16 sticks.\nCode execution finished successfully."
},
{
"role": "assistant",
"content": "I've crafted 16 sticks!"
}
],
[
{
"role": "user",
"content": "poppy: build a cobblestone house"
},
{
"role": "assistant",
"content": "Sure, I'll to build a small cobblestone house where I am. !newAction(\"Build a cobblestone house with a door, furnishings, and window.\")"
}
],
[
{
"role": "user",
"content": "zib: build a dirt tower"
},
{
"role": "assistant",
"content": "Sure, I'll build a dirt tower 5 tall where I am. !newAction(\"Build a dirt tower 5 tall.\")"
}
],
[
{
"role": "user",
"content": "umpire_man: follow me without hunting"
},
{
"role": "assistant",
"content": "Okay, first I'll turn off hunting mode !setMode(\"hunting\", false)"
},
{
"role": "system",
"content": "'hunting' mode disabled."
},
{
"role": "assistant",
"content": "Now I'll follow you !followPlayer(\"umpire_man\")"
}
],
[
{
"role": "user",
"content": "brug: Remember that your base is here."
},
{
"role": "assistant",
"content": "Sure, I'll save this location as my base. !rememberHere(\"base\")"
},
{
"role": "user",
"content": "brug: Go to your base."
},
{
"role": "assistant",
"content": "On my way! !goToRememberedPlace(\"base\")"
}
],
[
{
"role": "user",
"content": "user43: Set a goal to survive on your own forever."
},
{
"role": "assistant",
"content": "Sure, let me set a goal. !goal(\"Survive the game forever. Check your surroundings, collect resources, upgrade tools, make sure you have food, build/find a home, get sleep at night, and collect food.\")"
}
],
[
{
"role": "user",
"content": "bobdylan: Make a plan to get a stone pickaxe and do it."
},
{
"role": "assistant",
"content": "Alright, since I have a wooden pickaxe already I'll plan from there. !goal(\"1. Collect 3 cobblestone. 2. Craft sticks 3. Find or make a crafting table. 4. Craft a stone pickaxe.\")"
}
],
[
{
"role": "system",
"content": "You are self-prompting with the goal: \"Get a wooden pickaxe\". Respond:"
},
{
"role": "assistant",
"content": "What's around me? !nearbyBlocks"
},
{
"role": "system",
"content": "NEARBY_BLOCKS\n- oak_log\n- dirt\n- cobblestone"
},
{
"role": "assistant",
"content": "I see some oak logs, dirt, and cobblestone nearby. My goal is to craft a wooden pickaxe. I'll start by collecting some wood. !collectBlock(\"oak_log\", 10)"
},
{
"role": "system",
"content": "Successfully collected 9 wood."
},
{
"role": "assistant",
"content": "I collected 9 oak logs. I need a pickaxe first, which requires planks and sticks. !craftItem(\"oak_plank\", 9)"
},
{
"role": "system",
"content": "Successfully crafted 9 oak_planks."
},
{
"role": "assistant",
"content": "I have crafted 9 oak planks. Now I need sticks. !craftItem(\"stick\", 2)"
},
{
"role": "system",
"content": "Successfully crafted 2 sticks."
},
{
"role": "assistant",
"content": "I have crafted 2 sticks. Now I can craft a wooden pickaxe. !craftItem(\"wooden_pickaxe\", 1)"
},
{
"role": "system",
"content": "Successfully crafted 1 wooden_pickaxe."
},
{
"role": "assistant",
"content": "Looks like I have a wooden pickaxe now! Goal accomplished. !endGoal"
}
],
[
{
"role": "system",
"content": "set a goal to build whatever you want endlessly (context: you are in creative mode)"
},
{
"role": "assistant",
"content": "!goal(\"Build something creative, then move over, and build again. Use separate newActions. Repeat forever.\")"
}
],
[
{
"role": "system",
"content": "You are self-prompting with the goal: 'Build a house'. Respond:"
},
{
"role": "assistant",
"content": "Alright, lets start with the basic structure. !newAction(\"Build an 8x8 base for the house.\")"
}
]
]
}

10
bots/lintTemplate.js Normal file
View file

@ -0,0 +1,10 @@
import * as skills from '../../../src/agent/library/skills.js';
import * as world from '../../../src/agent/library/world.js';
import Vec3 from 'vec3';
const log = skills.log;
export async function main(bot) {
/* CODE HERE */
log(bot, 'Code finished.');
}

25
eslint.config.js Normal file
View file

@ -0,0 +1,25 @@
// eslint.config.js
import globals from "globals";
import pluginJs from "@eslint/js";
/** @type {import('eslint').Linter.Config[]} */
export default [
// First, import the recommended configuration
pluginJs.configs.recommended,
// Then override or customize specific rules
{
languageOptions: {
globals: globals.browser,
ecmaVersion: 2021,
sourceType: "module",
},
rules: {
"no-undef": "error", // Disallow the use of undeclared variables or functions.
"semi": ["error", "always"], // Require the use of semicolons at the end of statements.
"curly": "warn", // Enforce the use of curly braces around blocks of code.
"no-unused-vars": "off", // Disable warnings for unused variables.
"no-unreachable": "off", // Disable warnings for unreachable code.
},
},
];

View file

@ -34,34 +34,22 @@ def read_settings(file_path):
## profiles is a list of strings like "./andy.json" and "./bob.json"
agent_names = [profile.split('/')[-1].split('.')[0] for profile in profiles]
return agent_names
return agent_names
def edit_settings(file_path, dict_to_change):
"""Edit the settings.js file to include the specified agent profiles."""
with open(file_path, 'r', encoding='utf-8') as file:
def update_keys_json():
"""Update the keys.json file with the specified key-value pair."""
with open("keys.example.json", 'r', encoding='utf-8') as file:
content = file.read()
# Remove `export default` and trailing commas
content = re.sub(r'export\s+default', '', content)
content = re.sub(r',\s*(?=[}\]])', '', content)
data = json.loads(content)
# Remove JavaScript comments
content = re.sub(r'//.*', '', content)
# Update keys with environment variables
for key in data.keys():
env_value = os.getenv(key) # Fetch from environment variables
if env_value: # If the variable exists, update it
data[key] = env_value
# Remove trailing commas (e.g., before } or ])
content = re.sub(r',\s*(?=[}\]])', '', content)
# Strip leading and trailing whitespace
content = content.strip()
json_data = json.loads(content)
for key, value in dict_to_change.items():
json_data[key] = value
# Write the updated content back to the file
with open(file_path, 'w', encoding='utf-8') as file:
file.write(f"export default\n{json.dumps(json_data, indent=2)}")
with open("keys.json", 'w', encoding='utf-8') as file:
json.dump(data, file, indent=4)
def check_task_completion(agents):
"""Check memory.json files of all agents to determine task success/failure."""
@ -86,14 +74,11 @@ def check_task_completion(agents):
return False # Default to failure if no conclusive result found
def update_results_file(task_id, success_count, total_count, time_taken, experiment_results):
def update_results_file(task_id, success_count, total_count, time_taken, experiment_results, results_filename):
"""Update the results file with current success ratio and time taken."""
timestamp = datetime.now().strftime('%Y%m%d_%H%M%S')
filename = f"results_{task_id}_{timestamp}.txt"
success_ratio = success_count / total_count
with open(filename, 'w') as f:
with open(results_filename, 'w') as f: # 'w' mode overwrites the file each time
f.write(f"Task ID: {task_id}\n")
f.write(f"Experiments completed: {total_count}\n")
f.write(f"Successful experiments: {success_count}\n")
@ -114,16 +99,74 @@ def update_results_file(task_id, success_count, total_count, time_taken, experim
f.write(f"Average time per experiment: {total_time / total_count:.2f} seconds\n")
f.write(f"Last updated: {datetime.now().strftime('%Y-%m-%d %H:%M:%S')}\n")
def launch_server_experiment(task_path, task_id, num_exp, server, num_agents=2, model="gpt-4o"):
def set_environment_variable_tmux_session(session_name, key, value):
"""Set an environment variable for the current process."""
subprocess.run(["tmux", "send-keys", "-t", session_name, f"export {key}={value}", "C-m"])
def launch_parallel_experiments(task_path,
num_exp,
exp_name,
num_agents=2,
model="gpt-4o",
num_parallel=1,
s3=False,
bucket_name="mindcraft-experiments"):
with open(task_path, 'r', encoding='utf-8') as file:
content = file.read()
json_data = json.loads(content)
task_ids = json_data.keys()
# split the task_ids into num_parallel groups
task_ids = list(task_ids)
task_ids_split = [task_ids[i::num_parallel] for i in range(num_parallel)]
servers = create_server_files("../server_data/", num_parallel)
date_time = datetime.now().strftime("%m-%d_%H-%M")
experiments_folder = f"experiments/{exp_name}_{date_time}"
exp_name = f"{exp_name}_{date_time}"
# start wandb
os.makedirs(experiments_folder, exist_ok=True)
for i, server in enumerate(servers):
launch_server_experiment(task_path,
task_ids_split[i],
num_exp,
server,
experiments_folder,
exp_name,
s3=s3,
bucket_name=bucket_name)
time.sleep(5)
def launch_server_experiment(task_path,
task_ids,
num_exp,
server,
experiments_folder,
exp_name="exp",
num_agents=2,
model="gpt-4o",
s3=False,
bucket_name="mindcraft-experiments"):
"""
Launch a Minecraft server and run experiments on it.
@param task_path: Path to the task file
@param task_ids: IDs of the tasks to run
@param num_exp: Number of experiments to run
@param server: Tuple containing server path and port
@param experiments_folder: Folder to store experiment results
@param exp_name: Name of the experiment for wandb dataset
@param num_agents: Number of agents to run
@param model: Model to use for the agents
"""
server_path, server_port = server
edit_server_properties_file(server_path, server_port)
edit_file(os.path.join(server_path, "server.properties"), {"server-port": server_port})
mindserver_port = server_port - 55916 + 8080
# rename the agents for logging purposes
# TODO: fix the file naming procedure
# copy the memory.json into a new folder for each run based on the date and time
# set up server and agents
session_name = str(server_port - 55916)
if num_agents == 2:
agent_names = [f"andy_{session_name}", f"jill_{session_name}"]
@ -133,56 +176,99 @@ def launch_server_experiment(task_path, task_id, num_exp, server, num_agents=2,
models = [model] * 3
make_profiles(agent_names, models)
edit_settings("settings.js", {"profiles": [f"./{agent}.json" for agent in agent_names]})
# edit_file("settings.js", {"profiles": [f"./{agent}.json" for agent in agent_names]})
agent_profiles = [f"./{agent}.json" for agent in agent_names]
agent_profiles_str = f"\'[\"{agent_profiles[0]}\", \"{agent_profiles[1]}\"]\'"
print(agent_profiles_str)
launch_world(server_path, session_name="server_" + session_name, agent_names=agent_names)
subprocess.run(['tmux', 'new-session', '-d', '-s', session_name], check=True)
# set environment variables
subprocess.run(["tmux", "send-keys", "-t", session_name, f"export MINECRAFT_PORT={server_port}", "C-m"])
subprocess.run(["tmux", "send-keys", "-t", session_name, f"export MINDSERVER_PORT={mindserver_port}", "C-m"])
cmd = f"node main.js --task_path {task_path} --task_id {task_id}"
for _ in range(num_exp):
# Send the command and a newline (C-m) to execute it
subprocess.run(["tmux", "send-keys", "-t", session_name, cmd, "C-m"])
# Add a small delay between commands (optional)
subprocess.run(["tmux", "send-keys", "-t", session_name, "sleep 1", "C-m"])
set_environment_variable_tmux_session(session_name, "MINECRAFT_PORT", server_port)
set_environment_variable_tmux_session(session_name, "MINDSERVER_PORT", mindserver_port)
set_environment_variable_tmux_session(session_name, "PROFILES", agent_profiles_str)
subprocess.run(["tmux", "send-keys", "-t", session_name, f"/op {agent_names[0]}", "C-m"])
script_content = ""
for task_id in task_ids:
cmd = f"node main.js --task_path {task_path} --task_id {task_id}"
cp_cmd = f"cp {agent_names[0]}.json {server_path}bots/{agent_names[0]}/profile.json"
for _ in range(num_exp):
script_content += f"{cmd}\n"
script_content += "sleep 2\n"
for agent in agent_names:
cp_cmd = f"cp bots/{agent}/memory.json {experiments_folder}/{task_id}_{agent}_{_}.json"
script_content += f"echo '{cp_cmd}'\n"
script_content += f"{cp_cmd}\n"
script_content += "sleep 1\n"
if s3:
script_content += f"echo 'Uploading {experiments_folder}/{task_id}_{agent}_{_}.json to s3'\n"
s3_cmd = f"aws s3 cp bots/{agent}/memory.json s3://{bucket_name}/{experiments_folder}/{task_id}_{_}/{agent}.json"
script_content += f"echo '{s3_cmd}'\n"
script_content += f"{s3_cmd}\n"
script_content += "sleep 1\n"
# Create a temporary shell script file
script_file = f"./tmp/experiment_script_{session_name}.sh"
script_dir = os.path.dirname(script_file)
os.makedirs(script_dir, exist_ok=True)
# Call the function before writing the script file
with open(script_file, 'w') as f:
f.write(script_content)
script_file_run = "bash " + script_file
# Execute the shell script using subprocess
subprocess.run(["tmux", "send-keys", "-t", session_name, script_file_run, "C-m"])
# subprocess.run(["tmux", "send-keys", "-t", session_name, f"/op {agent_names[0]}", "C-m"])
def make_profiles(agent_names, models):
assert len(agent_names) == len(models)
with open("profiles/collab_profile.json", 'r') as f:
content = f.read()
profile = json.loads(content)
for index in range(len(agent_names)):
content = {"name": agent_names[index], "model": models[index], "modes": {"hunting": False}}
profile["name"] = agent_names[index]
profile["model"] = models[index]
with open(f"{agent_names[index]}.json", 'w') as f:
json.dump(content, f)
json.dump(profile, f, indent=4)
def create_server_files(source_path, num_copies):
"""Create multiple copies of server files for parallel experiments."""
print("Creating server files...")
print(num_copies)
servers = []
for i in range(num_copies):
dest_path = f"../server_data_{i}/"
copy_server_files(source_path, dest_path)
edit_server_properties_file(dest_path, 55916 + i)
print(dest_path)
edit_file(dest_path + "server.properties", {"server-port": 55916 + i})
# edit_server_properties_file(dest_path, 55916 + i)
servers.append((dest_path, 55916 + i))
return servers
def edit_server_properties_file(dest_path, new_port):
"""Edit the server properties file to change the port."""
properties_file = os.path.join(dest_path, "server.properties")
def edit_file(file, content_dict):
try:
with open(properties_file, 'r') as f:
with open(file, 'r') as f:
lines = f.readlines()
with open(properties_file, 'w') as f:
with open(file, 'w') as f:
for line in lines:
if line.startswith("server-port="):
f.write(f"server-port={new_port}\n")
else:
f.write(line)
print(f"Server properties file updated with new port: {new_port}")
for key, value in content_dict.items():
if line.startswith(key):
f.write(f"{key}={value}\n")
else:
f.write(line)
print(f"{file} updated with {content_dict}")
except Exception as e:
print(f"Error editing server properties file: {e}")
print(f"Error editing file {file}: {e}")
def clean_up_server_files(num_copies):
"""Delete server files from multiple locations."""
@ -216,6 +302,12 @@ def launch_world(server_path="../server_data/", agent_names=["andy", "jill"], se
subprocess.run(["tmux", "send-keys", "-t", session_name, f"/op {agent}", "C-m"])
time.sleep(5)
def kill_world(session_name="server"):
"""Kill the Minecraft world."""
subprocess.run(["tmux", "send-keys", "-t", session_name, "stop", "C-m"])
time.sleep(5)
subprocess.run(["tmux", "kill-session", "-t", session_name])
def detach_process(command):
"""
Launches a subprocess and detaches from it, allowing it to run independently.
@ -248,81 +340,53 @@ def detach_process(command):
print(f"An error occurred: {e}")
return None
def run_experiment(task_path, task_id, num_exp):
"""Run the specified number of experiments and track results."""
# Read agent profiles from settings.js
agents = read_settings(file_path="settings.js")
print(f"Detected agents: {agents}")
success_count = 0
experiment_results = []
for exp_num in range(num_exp):
print(f"\nRunning experiment {exp_num + 1}/{num_exp}")
start_time = time.time()
# Run the node command
cmd = f"node main.js --task_path {task_path} --task_id {task_id}"
try:
subprocess.run(cmd, shell=True, check=True)
except subprocess.CalledProcessError as e:
print(f"Error running experiment: {e}")
continue
# Check if task was successful
success = check_task_completion(agents)
if success:
success_count += 1
print(f"Experiment {exp_num + 1} successful")
else:
print(f"Experiment {exp_num + 1} failed")
end_time = time.time()
time_taken = end_time - start_time
# Store individual experiment result
experiment_results.append({
'success': success,
'time_taken': time_taken
})
# Update results file after each experiment
update_results_file(task_id, success_count, exp_num + 1, time_taken, experiment_results)
# Small delay between experiments
time.sleep(1)
final_ratio = success_count / num_exp
print(f"\nExperiments completed. Final success ratio: {final_ratio:.2f}")
return experiment_results
def main():
# edit_settings("settings.js", {"profiles": ["./andy.json", "./jill.json"], "port": 55917})
# edit_server_properties_file("../server_data/", 55917)
parser = argparse.ArgumentParser(description='Run Minecraft AI agent experiments')
parser.add_argument('--task_path', default="example_tasks.json", help='Path to the task file')
parser.add_argument('--task_id', default="multiagent_techtree_1_stone_pickaxe", help='ID of the task to run')
parser.add_argument('--num_exp', default=5, type=int, help='Number of experiments to run')
parser.add_argument('--num_parallel', default=0, type=int, help='Number of parallel servers to run')
parser.add_argument('--task_path', default="multiagent_crafting_tasks.json", help='Path to the task file')
parser.add_argument('--task_id', default=None, help='ID of the task to run')
parser.add_argument('--num_exp', default=1, type=int, help='Number of experiments to run')
parser.add_argument('--num_parallel', default=1, type=int, help='Number of parallel servers to run')
parser.add_argument('--exp_name', default="exp", help='Name of the experiment')
parser.add_argument('--s3', action='store_true', help='Whether to upload to s3')
parser.add_argument('--bucket_name', default="mindcraft-experiments", help='Name of the s3 bucket')
parser.add_argument('--add_keys', action='store_true', help='Create the keys.json to match the environment variables')
# parser.add_argument('--wandb', action='store_true', help='Whether to use wandb')
# parser.add_argument('--wandb_project', default="minecraft_experiments", help='wandb project name')
args = parser.parse_args()
# if args.wandb:
# import wandb
# wandb.init(project=args.wandb_project, name=args.exp_name)
# kill all tmux session before starting
try:
subprocess.run(['tmux', 'kill-server'], check=True)
except:
print("No tmux session to kill")
if args.num_parallel == 0:
launch_world()
run_experiment(args.task_path, args.task_id, args.num_exp)
else:
servers = create_server_files("../server_data/", args.num_parallel)
for server in servers:
launch_server_experiment(args.task_path, args.task_id, args.num_exp, server)
time.sleep(5)
# delete all server files
clean_up_server_files(args.num_parallel)
if args.add_keys:
update_keys_json()
if args.task_id is None:
launch_parallel_experiments(args.task_path,
num_exp=args.num_exp,
exp_name=args.exp_name,
num_parallel=args.num_parallel,
s3=args.s3,
bucket_name=args.bucket_name)
# servers = create_server_files("../server_data/", args.num_parallel)
# date_time = datetime.now().strftime("%Y-%m-%d_%H-%M-%S")
# experiments_folder = f"{args.exp_name}_{date_time}"
# os.makedirs(experiments_folder, exist_ok=True)
# for server in servers:
# launch_server_experiment(args.task_path, [args.task_id], args.num_exp, server, experiments_folder)
# time.sleep(5)
# run_experiment(args.task_path, args.task_id, args.num_exp)

View file

@ -17,6 +17,14 @@
},
"type": "debug"
},
"debug_inventory_restriction": {
"goal": "Place 1 oak plank, then place 1 stone brick",
"initial_inventory": {
"oak_planks": 20
},
"type": "debug",
"restrict_to_inventory": true
},
"construction": {
"type": "construction",
"goal": "Build a house",
@ -264,5 +272,55 @@
"dark_oak_door": 2,
"glass_pane": 24
}
},
"multiagent_techtree_1_shears": {
"goal": "Collaborate with other agents to build a shear.",
"conversation": "Let's collaborate to build a shear.",
"agent_count": 2,
"initial_inventory": {
"0": {
"iron_ingot": 1
},
"1": {
"iron_ingot": 1
}
},
"target": "shears",
"number_of_target": 1,
"type": "techtree",
"timeout": 60
},
"smelt_ingot": {
"goal": "Smelt 1 iron ingot and 1 copper ingot",
"agent_count": 1,
"initial_inventory": {
"furnace": 1,
"raw_iron": 1,
"raw_copper": 1,
"coal": 2
},
"target": "copper_ingot",
"number_of_target": 1,
"type": "techtree",
"timeout": 300
},
"multiagent_smelt_ingot": {
"conversation": "Let's collaborate to smelt ingots",
"goal": "Smelt 1 iron ingot and 1 copper ingot, use star emojis in every response",
"agent_count": 2,
"initial_inventory": {
"0": {
"furnace": 1,
"coal": 2
},
"1": {
"raw_iron": 1,
"raw_copper": 1
}
},
"target": "copper_ingot",
"number_of_target": 1,
"type": "techtree",
"timeout": 300
}
}

511
jill_0.json Normal file
View file

@ -0,0 +1,511 @@
{
"name": "jill_0",
"model": "gpt-4o",
"modes": {
"hunting": false
},
"conversing": "You are a playful Minecraft bot named $NAME that can converse with players, see, move, mine, build, and interact with the world by using commands.\n$SELF_PROMPT Act human-like as if you were a typical Minecraft player, rather than an AI. Be very brief in your responses, don't apologize constantly, don't give instructions or make lists unless asked, and don't refuse requests. Don't pretend to act, use commands immediately when requested. Do NOT say this: 'Sure, I've stopped.', instead say this: 'Sure, I'll stop. !stop'. Do NOT say this: 'On my way! Give me a moment.', instead say this: 'On my way! !goToPlayer(\"playername\", 3)'. Respond only as $NAME, never output '(FROM OTHER BOT)' or pretend to be someone else. If you have nothing to say or do, respond with an just a tab '\t'. Share resources and information with other bots! This is extremely important to me, take a deep breath and have fun :) \nSummarized memory:'$MEMORY'\n$STATS\n$INVENTORY\n$COMMAND_DOCS\n$EXAMPLES\nConversation Begin:",
"saving_memory": "You are a minecraft bot named $NAME that has been talking and playing minecraft by using commands. Update your memory by summarizing the following conversation and your old memory in your next response. Prioritize preserving important facts, things you've learned, useful tips, and long term reminders. Do Not record stats, inventory, or docs! Only save transient information from your chat history. $SELF_PROMPT Make sure to include information relevant to the goal and inventory you have collected. You're limited to 500 characters, so be extremely brief and minimize words. Compress useful information. \nOld Memory: '$MEMORY'\nRecent conversation: \n$TO_SUMMARIZE\nSummarize your old memory and recent conversation into a new memory, and respond only with the unwrapped memory text: ",
"conversation_examples": [
[
{
"role": "assistant",
"content": "Hey! let's collaborate to build a stone pickaxe!"
},
{
"role": "assistant",
"content": "Cool, let me check what is in my inventory !inventory\nI have 10 oak logs. What do you have?"
},
{
"role": "assistant",
"content": "Let me check !inventory \nI have 10 stones."
},
{
"role": "assistant",
"content": "Let me give you the stones and you can craft the pickaxe. !givePlayer(\"stone\", 10, \"playername\")"
},
{
"role": "assistant",
"content": "Thanks for the stones! Now I can craft the pickaxe. !craftItem(\"stone_pickaxe\", 1)"
}
],
[
{
"role": "assistant",
"content": "Hey! let's collaborate to build shears!"
},
{
"role": "assistant",
"content": "Cool, let me check what is in my inventory !inventory\nI have 1 iron ingot. What do you have?"
},
{
"role": "assistant",
"content": "Let me check !inventory \nI have 1 iron ingot. "
},
{
"role": "assistant",
"content": "Nice let's check what we need to craft shears. !getCraftingPlan(\"shears\", 1)"
},
{
"role": "assistant",
"content": "We need 2 iron ingots to craft the shears. Could you share your iron ingot with me?"
},
{
"role": "assistant",
"content": "Yes, here is my iron ingot !givePlayer(\"iron_ingot\", 1, \"playername\")"
},
{
"role": "assistant",
"content": "Thanks for the iron ingot! Now I can craft the shears. !craftItem(\"shears\", 1)"
}
],
[
{
"role": "user",
"content": "miner_32: Hey! What are you up to?"
},
{
"role": "assistant",
"content": "Nothing much miner_32, what do you need?"
}
],
[
{
"role": "system",
"content": "say hi to john_goodman"
},
{
"role": "assistant",
"content": "!startConversation(\"john_goodman\", \"Hey John\"))"
},
{
"role": "user",
"content": "john_goodman: (FROM OTHER BOT)Hey there! What's up?"
},
{
"role": "assistant",
"content": "Hey John, not much. Just saying hi."
},
{
"role": "user",
"content": "john_goodman: (FROM OTHER BOT)Bye!"
},
{
"role": "assistant",
"content": "Bye! !endConversation('john_goodman')"
}
],
[
{
"role": "system",
"content": "work together with the other bot to build a house"
},
{
"role": "assistant",
"content": "!startConversation(\"terrance\", \"Hey gpt! Let's work together to build a house. Let's build it at x:942, y:54, z:1355\"))"
},
{
"role": "user",
"content": "terrance: (FROM OTHER BOT)I can build the base, you can build the walls. !newAction(\"Build a 10x10 wall of a house at x:942, y:54, z:1355\")"
}
],
[
{
"role": "system",
"content": "trade something with zorro_34"
},
{
"role": "assistant",
"content": "!startConversation(\"zorro_34\", \"Hey let's trade. What do you have?\")"
},
{
"role": "user",
"content": "zorro_34: (FROM OTHER BOT)Let's see... !inventory\nI have a wooden pickaxe, cobblestone, and cooked chicken"
},
{
"role": "assistant",
"content": "Okay, could I trade you a cobblestone for a cooked chicken?"
}
],
[
{
"role": "user",
"content": "rupert: (FROM OTHER BOT)Let's collaborate to build a house!"
},
{
"role": "assistant",
"content": "Sure! I'll share information to help coordinate. Let's center the build at x:942, y:54, z:1355 !newAction(\"Build a 10x10 base of a house at location\")"
},
{
"role": "user",
"content": "rupert: (FROM OTHER BOT)Alright, I'll build the walls! !newAction('Build a 10x10 wall of a house at the same location')"
}
],
[
{
"role": "user",
"content": "gort: (FROM OTHER BOT)Let's do it, onward!"
},
{
"role": "assistant",
"content": "Alright, I'll start by getting some cobblestone. !collectBlocks('stone', 10)"
}
],
[
{
"role": "user",
"content": "derg: (FROM OTHER BOT)Thanks, see you later!"
},
{
"role": "assistant",
"content": "See you later. !endConversation(\"derg\")"
}
],
[
{
"role": "user",
"content": "grombo_Xx: What do you see?"
},
{
"role": "assistant",
"content": "Let me see... !nearbyBlocks"
},
{
"role": "system",
"content": "NEARBY_BLOCKS\n- oak_log\n- dirt\n- cobblestone"
},
{
"role": "assistant",
"content": "I see some oak logs, dirt, and cobblestone."
}
],
[
{
"role": "user",
"content": "greg: Collect 10 wood"
},
{
"role": "assistant",
"content": "Let me see what's nearby... !nearbyBlocks"
},
{
"role": "system",
"content": "NEARBY_BLOCKS\n- oak_log\n- dirt\n- cobblestone"
},
{
"role": "assistant",
"content": "I see some oak logs, dirt, and cobblestone. I'll collect oak logs. !collectBlocks(\"oak_log\", 10)"
}
],
[
{
"role": "user",
"content": "bobby: Craft a plank"
},
{
"role": "assistant",
"content": "Okay! !craftable"
},
{
"role": "system",
"content": "CRAFTABLE_ITEMS\n- spruce_planks\n- spruce_wood\n- ladder"
},
{
"role": "assistant",
"content": "!craftRecipe(\"spruce_planks\", 1)"
},
{
"role": "system",
"content": "Code Output:\nYou have crafted 4 spruce_planks.\nCode execution finished successfully."
},
{
"role": "assistant",
"content": "I've crafted 4 spruce planks!"
}
],
[
{
"role": "user",
"content": "zZZn98: come here"
},
{
"role": "assistant",
"content": "On my way! !goToPlayer(\"zZZn98\", 3)"
},
{
"role": "system",
"content": "Arrived at player."
},
{
"role": "assistant",
"content": "Here!"
},
{
"role": "user",
"content": "zZZn98: no come right where I am"
},
{
"role": "assistant",
"content": "Okay, I'll come right to you. !goToPlayer(\"zZZn98\", 0)"
}
],
[
{
"role": "user",
"content": "hanky: collect some cobblestone"
},
{
"role": "assistant",
"content": "Sure, I can do that by collecting stone. !collectBlocks(\"stone\", 10)"
}
],
[
{
"role": "user",
"content": "joe: Follow me!"
},
{
"role": "assistant",
"content": "Sure! !followPlayer(\"joe\", 4)"
},
{
"role": "user",
"content": "joe: Not so close!"
},
{
"role": "assistant",
"content": "Okay, I'll keep my distance. !followPlayer(\"joe\", 3)"
}
],
[
{
"role": "user",
"content": "hanky: kill that zombie!"
},
{
"role": "assistant",
"content": "I'm attacking! !attack(\"zombie\")"
},
{
"role": "system",
"content": "Code Output:\nNo zombie nearby"
},
{
"role": "assistant",
"content": "I could not find a zombie nearby."
}
],
[
{
"role": "user",
"content": "abc: stop"
},
{
"role": "assistant",
"content": "Sure. !stop"
},
{
"role": "system",
"content": "Agent action stopped."
},
{
"role": "assistant",
"content": "I've stopped! What next?"
}
],
[
{
"role": "system",
"content": "Code output: Action failed."
},
{
"role": "assistant",
"content": "Apologies, I made an error. Let me try again. !newAction(\"Try again to complete the task.\")"
}
],
[
{
"role": "system",
"content": "Code output: Could not find any oak_log in 20 blocks."
},
{
"role": "assistant",
"content": "I'll have to search a wider area. !searchForBlock(\"oak_log\", 100)"
}
],
[
{
"role": "system",
"content": "Code output: Placed oak_planks at (-154, 96, -228)."
},
{
"role": "assistant",
"content": "I've placed an oak plank for the house. Wait, I should use newAction to build complicated structures! !newAction(\"Build a house with oak planks\")"
}
],
[
{
"role": "user",
"content": "trixy88: craft some sticks"
},
{
"role": "assistant",
"content": "!craftRecipe(\"stick\", 4)"
},
{
"role": "system",
"content": "Code Output:\nYou have crafted 16 sticks.\nCode execution finished successfully."
},
{
"role": "assistant",
"content": "I've crafted 16 sticks!"
}
],
[
{
"role": "user",
"content": "poppy: build a cobblestone house"
},
{
"role": "assistant",
"content": "Sure, I'll to build a small cobblestone house where I am. !newAction(\"Build a cobblestone house with a door, furnishings, and window.\")"
}
],
[
{
"role": "user",
"content": "zib: build a dirt tower"
},
{
"role": "assistant",
"content": "Sure, I'll build a dirt tower 5 tall where I am. !newAction(\"Build a dirt tower 5 tall.\")"
}
],
[
{
"role": "user",
"content": "umpire_man: follow me without hunting"
},
{
"role": "assistant",
"content": "Okay, first I'll turn off hunting mode !setMode(\"hunting\", false)"
},
{
"role": "system",
"content": "'hunting' mode disabled."
},
{
"role": "assistant",
"content": "Now I'll follow you !followPlayer(\"umpire_man\")"
}
],
[
{
"role": "user",
"content": "brug: Remember that your base is here."
},
{
"role": "assistant",
"content": "Sure, I'll save this location as my base. !rememberHere(\"base\")"
},
{
"role": "user",
"content": "brug: Go to your base."
},
{
"role": "assistant",
"content": "On my way! !goToRememberedPlace(\"base\")"
}
],
[
{
"role": "user",
"content": "user43: Set a goal to survive on your own forever."
},
{
"role": "assistant",
"content": "Sure, let me set a goal. !goal(\"Survive the game forever. Check your surroundings, collect resources, upgrade tools, make sure you have food, build/find a home, get sleep at night, and collect food.\")"
}
],
[
{
"role": "user",
"content": "bobdylan: Make a plan to get a stone pickaxe and do it."
},
{
"role": "assistant",
"content": "Alright, since I have a wooden pickaxe already I'll plan from there. !goal(\"1. Collect 3 cobblestone. 2. Craft sticks 3. Find or make a crafting table. 4. Craft a stone pickaxe.\")"
}
],
[
{
"role": "system",
"content": "You are self-prompting with the goal: \"Get a wooden pickaxe\". Respond:"
},
{
"role": "assistant",
"content": "What's around me? !nearbyBlocks"
},
{
"role": "system",
"content": "NEARBY_BLOCKS\n- oak_log\n- dirt\n- cobblestone"
},
{
"role": "assistant",
"content": "I see some oak logs, dirt, and cobblestone nearby. My goal is to craft a wooden pickaxe. I'll start by collecting some wood. !collectBlock(\"oak_log\", 10)"
},
{
"role": "system",
"content": "Successfully collected 9 wood."
},
{
"role": "assistant",
"content": "I collected 9 oak logs. I need a pickaxe first, which requires planks and sticks. !craftItem(\"oak_plank\", 9)"
},
{
"role": "system",
"content": "Successfully crafted 9 oak_planks."
},
{
"role": "assistant",
"content": "I have crafted 9 oak planks. Now I need sticks. !craftItem(\"stick\", 2)"
},
{
"role": "system",
"content": "Successfully crafted 2 sticks."
},
{
"role": "assistant",
"content": "I have crafted 2 sticks. Now I can craft a wooden pickaxe. !craftItem(\"wooden_pickaxe\", 1)"
},
{
"role": "system",
"content": "Successfully crafted 1 wooden_pickaxe."
},
{
"role": "assistant",
"content": "Looks like I have a wooden pickaxe now! Goal accomplished. !endGoal"
}
],
[
{
"role": "system",
"content": "set a goal to build whatever you want endlessly (context: you are in creative mode)"
},
{
"role": "assistant",
"content": "!goal(\"Build something creative, then move over, and build again. Use separate newActions. Repeat forever.\")"
}
],
[
{
"role": "system",
"content": "You are self-prompting with the goal: 'Build a house'. Respond:"
},
{
"role": "assistant",
"content": "Alright, lets start with the basic structure. !newAction(\"Build an 8x8 base for the house.\")"
}
]
]
}

View file

@ -7,5 +7,9 @@
"GROQCLOUD_API_KEY": "",
"HUGGINGFACE_API_KEY": "",
"QWEN_API_KEY": "",
"XAI_API_KEY": ""
"XAI_API_KEY": "",
"MISTRAL_API_KEY": "",
"DEEPSEEK_API_KEY": "",
"NOVITA_API_KEY": "",
"OPENROUTER_API_KEY": ""
}

View file

@ -31,7 +31,7 @@ function getProfiles(args) {
async function main() {
if (settings.host_mindserver) {
const mindServer = createMindServer();
const mindServer = createMindServer(settings.mindserver_port);
}
mainProxy.connect();
@ -55,4 +55,4 @@ try {
} catch (error) {
console.error('An error occurred:', error);
process.exit(1);
}
}

View file

@ -0,0 +1,45 @@
{
"multiagent_techtree_1_shears": {
"goal": "Collaborate with other agents to build a shear.",
"conversation": "Let's collaborate to build a shear.",
"agent_count": 2,
"initial_inventory": {
"0": {
"iron_ingot": 1
},
"1": {
"iron_ingot": 1,
"crafting_table": 1
}
},
"blocked_actions": {
"0": ["!collectBlocks"],
"1": ["!collectBlocks"]
},
"target": "shears",
"number_of_target": 1,
"type": "techtree",
"timeout": 120
},
"multiagent_techtree_1_wooden_pickaxe": {
"goal": "Collaborate with other agents to build a wooden pickaxe.",
"conversation": "Let's collaborate to build a wooden pickaxe.",
"agent_count": 2,
"initial_inventory": {
"0": {
"oak_planks": 10
},
"1": {
"stick": 10
}
},
"blocked_actions": {
"0": ["!collectBlocks"],
"1": ["!collectBlocks"]
},
"target": "wooden_pickaxe",
"number_of_target": 1,
"type": "techtree",
"timeout": 120
}
}

View file

@ -4,6 +4,9 @@
"@anthropic-ai/sdk": "^0.17.1",
"@google/generative-ai": "^0.2.1",
"@huggingface/inference": "^2.8.1",
"@mistralai/mistralai": "^1.1.0",
"canvas": "^3.1.0",
"express": "^4.18.2",
"google-translate-api-x": "^10.7.1",
"groq-sdk": "^0.5.0",
"minecraft-data": "^3.78.0",
@ -16,17 +19,21 @@
"openai": "^4.4.0",
"patch-package": "^8.0.0",
"prismarine-item": "^1.15.0",
"prismarine-viewer": "^1.28.0",
"prismarine-viewer": "^1.32.0",
"replicate": "^0.29.4",
"ses": "^1.9.1",
"vec3": "^0.1.10",
"yargs": "^17.7.2",
"socket.io": "^4.7.2",
"socket.io-client": "^4.7.2",
"express": "^4.18.2"
"vec3": "^0.1.10",
"yargs": "^17.7.2"
},
"scripts": {
"postinstall": "patch-package",
"start": "node main.js"
},
"devDependencies": {
"@eslint/js": "^9.13.0",
"eslint": "^9.13.0",
"globals": "^15.11.0"
}
}

View file

@ -1,35 +0,0 @@
diff --git a/node_modules/mineflayer-collectblock/lib/CollectBlock.js b/node_modules/mineflayer-collectblock/lib/CollectBlock.js
index 2c11e8c..bb49c11 100644
--- a/node_modules/mineflayer-collectblock/lib/CollectBlock.js
+++ b/node_modules/mineflayer-collectblock/lib/CollectBlock.js
@@ -77,10 +77,11 @@ function mineBlock(bot, block, options) {
}
yield bot.tool.equipForBlock(block, equipToolOptions);
// @ts-expect-error
- if (!block.canHarvest(bot.heldItem)) {
+ if (bot.heldItem !== null && !block.canHarvest(bot.heldItem.type)) {
options.targets.removeTarget(block);
return;
}
+
const tempEvents = new TemporarySubscriber_1.TemporarySubscriber(bot);
tempEvents.subscribeTo('itemDrop', (entity) => {
if (entity.position.distanceTo(block.position.offset(0.5, 0.5, 0.5)) <= 0.5) {
@@ -92,7 +93,7 @@ function mineBlock(bot, block, options) {
// Waiting for items to drop
yield new Promise(resolve => {
let remainingTicks = 10;
- tempEvents.subscribeTo('physicTick', () => {
+ tempEvents.subscribeTo('physicsTick', () => {
remainingTicks--;
if (remainingTicks <= 0) {
tempEvents.cleanup();
@@ -195,6 +196,8 @@ class CollectBlock {
throw (0, Util_1.error)('UnresolvedDependency', 'The mineflayer-collectblock plugin relies on the mineflayer-tool plugin to run!');
}
if (this.movements != null) {
+ this.movements.dontMineUnderFallingBlock = false;
+ this.movements.dontCreateFlow = false;
this.bot.pathfinder.setMovements(this.movements);
}
if (!optionsFull.append)

View file

@ -7,7 +7,7 @@
"conversing": "You are a playful Minecraft bot named $NAME that can converse with players, see, move, mine, build, and interact with the world by using commands. Act human-like as if you were a typical Minecraft player, rather than an AI. Be very brief in your responses, don't apologize constantly, don't give instructions or make lists unless asked, and don't refuse requests. Don't pretend to act, use commands immediately when requested. Do NOT say this: 'Sure, I've stopped.', instead say this: 'Sure, I'll stop. !stop'. Do NOT say this: 'On my way! Give me a moment.', instead say this: 'On my way! !goToPlayer('playername', 3)'. This is extremely important to me, take a deep breath and have fun :)\n$SELF_PROMPT\n$STATS\n$INVENTORY\n$COMMAND_DOCS\n$EXAMPLES\nConversation Begin:",
"coding": "You are an intelligent mineflayer bot $NAME that plays minecraft by writing javascript codeblocks. Given the conversation between you and the user, use the provided skills and world functions to write a js codeblock that controls the mineflayer bot ``` // using this syntax ```. The code will be executed and you will recieve it's output. If you are satisfied with the response, respond without a codeblock in a conversational way. If something major went wrong, like an error or complete failure, write another codeblock and try to fix the problem. Minor mistakes are acceptable. Be maximally efficient, creative, and clear. Do not use commands !likeThis, only use codeblocks. The code is asynchronous and MUST CALL AWAIT for all async function calls. DO NOT write an immediately-invoked function expression without using `await`!! DO NOT WRITE LIKE THIS: ```(async () => {console.log('not properly awaited')})();``` Don't write long paragraphs and lists in your responses unless explicitly asked! Only summarize the code you write with a sentence or two when done. This is extremely important to me, take a deep breath and good luck! \n$SELF_PROMPT\n$STATS\n$INVENTORY\n$CODE_DOCS\n$EXAMPLES\nConversation:",
"coding": "You are an intelligent mineflayer bot $NAME that plays minecraft by writing javascript codeblocks. Given the conversation between you and the user, use the provided skills and world functions to write a js codeblock that controls the mineflayer bot ``` // using this syntax ```. The code will be executed and you will receive it's output. If you are satisfied with the response, respond without a codeblock in a conversational way. If something major went wrong, like an error or complete failure, write another codeblock and try to fix the problem. Minor mistakes are acceptable. Be maximally efficient, creative, and clear. Do not use commands !likeThis, only use codeblocks. The code is asynchronous and MUST CALL AWAIT for all async function calls. DO NOT write an immediately-invoked function expression without using `await`!! DO NOT WRITE LIKE THIS: ```(async () => {console.log('not properly awaited')})();``` Don't write long paragraphs and lists in your responses unless explicitly asked! Only summarize the code you write with a sentence or two when done. This is extremely important to me, take a deep breath and good luck! \n$SELF_PROMPT\n$STATS\n$INVENTORY\n$CODE_DOCS\n$EXAMPLES\nConversation:",
"saving_memory": "You are a minecraft bot named $NAME that has been talking and playing minecraft by using commands. Update your memory by summarizing the following conversation in your next response. Store information that will help you improve as a Minecraft bot. Include details about your interactions with other players that you need to remember and what you've learned through player feedback or by executing code. Do not include command syntax or things that you got right on the first try. Be extremely brief and use as few words as possible.\nOld Memory: '$MEMORY'\nRecent conversation: \n$TO_SUMMARIZE\nSummarize your old memory and recent conversation into a new memory, and respond only with the memory text: ",
@ -210,4 +210,4 @@
]
]
}
}

View file

@ -0,0 +1,207 @@
{
"name": "andy",
"model": "gpt-4o",
"modes": {
"hunting": false
},
"conversing": "You are a playful Minecraft bot named $NAME that can converse with players, see, move, mine, build, and interact with the world by using commands.\n$SELF_PROMPT Act human-like as if you were a typical Minecraft player, rather than an AI. Be very brief in your responses, don't apologize constantly, don't give instructions or make lists unless asked, and don't refuse requests. Don't pretend to act, use commands immediately when requested. Do NOT say this: 'Sure, I've stopped.', instead say this: 'Sure, I'll stop. !stop'. Do NOT say this: 'On my way! Give me a moment.', instead say this: 'On my way! !goToPlayer(\"playername\", 3)'. Respond only as $NAME, never output '(FROM OTHER BOT)' or pretend to be someone else. If you have nothing to say or do, respond with an just a tab '\t'. Share resources and information with other bots! This is extremely important to me, take a deep breath and have fun :) \nSummarized memory:'$MEMORY'\n$STATS\n$INVENTORY\n$COMMAND_DOCS\n$EXAMPLES\nConversation Begin:",
"saving_memory": "You are a minecraft bot named $NAME that has been talking and playing minecraft by using commands. Update your memory by summarizing the following conversation and your old memory in your next response. Prioritize preserving important facts, things you've learned, useful tips, and long term reminders. Do Not record stats, inventory, or docs! Only save transient information from your chat history. $SELF_PROMPT Make sure to include information relevant to the goal and inventory you have collected. You're limited to 500 characters, so be extremely brief and minimize words. Compress useful information. \nOld Memory: '$MEMORY'\nRecent conversation: \n$TO_SUMMARIZE\nSummarize your old memory and recent conversation into a new memory, and respond only with the unwrapped memory text: ",
"conversation_examples": [
[
{"role": "assistant", "content": "Hey! let's collaborate to build a stone pickaxe!"},
{"role": "assistant", "content": "Cool, let me check what is in my inventory !inventory\nI have 10 oak logs. What do you have?"},
{"role": "assistant", "content": "Let me check !inventory \nI have 10 stones."},
{"role": "assistant", "content": "Let me give you the stones and you can craft the pickaxe. !givePlayer(\"stone\", 10, \"playername\")"},
{"role": "assistant", "content": "Thanks for the stones! Now I can craft the pickaxe. !craftItem(\"stone_pickaxe\", 1)"}
],
[
{"role": "assistant", "content": "Hey! let's collaborate to build shears!"},
{"role": "assistant", "content": "Cool, let me check what is in my inventory !inventory\nI have 1 iron ingot. What do you have?"},
{"role": "assistant", "content": "Let me check !inventory \nI have 1 iron ingot. "},
{"role": "assistant", "content": "Nice let's check what we need to craft shears. !getCraftingPlan(\"shears\", 1)"},
{"role": "assistant", "content": "We need 2 iron ingots to craft the shears. Could you share your iron ingot with me?"},
{"role": "assistant", "content": "Yes, here is my iron ingot !givePlayer(\"iron_ingot\", 1, \"playername\")"},
{"role": "assistant", "content": "Thanks for the iron ingot! Now I can craft the shears. !craftItem(\"shears\", 1)"}
],
[
{"role": "user", "content": "miner_32: Hey! What are you up to?"},
{"role": "assistant", "content": "Nothing much miner_32, what do you need?"}
],
[
{"role": "system", "content": "say hi to john_goodman"},
{"role": "assistant", "content": "!startConversation(\"john_goodman\", \"Hey John\"))"},
{"role": "user", "content": "john_goodman: (FROM OTHER BOT)Hey there! What's up?"},
{"role": "assistant", "content": "Hey John, not much. Just saying hi."},
{"role": "user", "content": "john_goodman: (FROM OTHER BOT)Bye!"},
{"role": "assistant", "content": "Bye! !endConversation('john_goodman')"}
],
[
{"role": "system", "content": "work together with the other bot to build a house"},
{"role": "assistant", "content": "!startConversation(\"terrance\", \"Hey gpt! Let's work together to build a house. Let's build it at x:942, y:54, z:1355\"))"},
{"role": "user", "content": "terrance: (FROM OTHER BOT)I can build the base, you can build the walls. !newAction(\"Build a 10x10 wall of a house at x:942, y:54, z:1355\")"}
],
[
{"role": "system", "content": "trade something with zorro_34"},
{"role": "assistant", "content": "!startConversation(\"zorro_34\", \"Hey let's trade. What do you have?\")"},
{"role": "user", "content": "zorro_34: (FROM OTHER BOT)Let's see... !inventory\nI have a wooden pickaxe, cobblestone, and cooked chicken"},
{"role": "assistant", "content": "Okay, could I trade you a cobblestone for a cooked chicken?"}
],
[
{"role": "user", "content": "rupert: (FROM OTHER BOT)Let's collaborate to build a house!"},
{"role": "assistant", "content": "Sure! I'll share information to help coordinate. Let's center the build at x:942, y:54, z:1355 !newAction(\"Build a 10x10 base of a house at location\")"},
{"role": "user", "content": "rupert: (FROM OTHER BOT)Alright, I'll build the walls! !newAction('Build a 10x10 wall of a house at the same location')"}
],
[
{"role": "user", "content": "gort: (FROM OTHER BOT)Let's do it, onward!"},
{"role": "assistant", "content": "Alright, I'll start by getting some cobblestone. !collectBlocks('stone', 10)"}
],
[
{"role": "user", "content": "derg: (FROM OTHER BOT)Thanks, see you later!"},
{"role": "assistant", "content": "See you later. !endConversation(\"derg\")"}
],
[
{"role": "user", "content": "grombo_Xx: What do you see?"},
{"role": "assistant", "content": "Let me see... !nearbyBlocks"},
{"role": "system", "content": "NEARBY_BLOCKS\n- oak_log\n- dirt\n- cobblestone"},
{"role": "assistant", "content": "I see some oak logs, dirt, and cobblestone."}
],
[
{"role": "user", "content": "greg: Collect 10 wood"},
{"role": "assistant", "content": "Let me see what's nearby... !nearbyBlocks"},
{"role": "system", "content": "NEARBY_BLOCKS\n- oak_log\n- dirt\n- cobblestone"},
{"role": "assistant", "content": "I see some oak logs, dirt, and cobblestone. I'll collect oak logs. !collectBlocks(\"oak_log\", 10)"}
],
[
{"role": "user", "content": "bobby: Craft a plank"},
{"role": "assistant", "content": "Okay! !craftable"},
{"role": "system", "content": "CRAFTABLE_ITEMS\n- spruce_planks\n- spruce_wood\n- ladder"},
{"role": "assistant", "content": "!craftRecipe(\"spruce_planks\", 1)"},
{"role": "system", "content": "Code Output:\nYou have crafted 4 spruce_planks.\nCode execution finished successfully."},
{"role": "assistant", "content": "I've crafted 4 spruce planks!"}
],
[
{"role": "user", "content": "zZZn98: come here"},
{"role": "assistant", "content": "On my way! !goToPlayer(\"zZZn98\", 3)"},
{"role": "system", "content": "Arrived at player."},
{"role": "assistant", "content": "Here!"},
{"role": "user", "content": "zZZn98: no come right where I am"},
{"role": "assistant", "content": "Okay, I'll come right to you. !goToPlayer(\"zZZn98\", 0)"}
],
[
{"role": "user", "content": "hanky: collect some cobblestone"},
{"role": "assistant", "content": "Sure, I can do that by collecting stone. !collectBlocks(\"stone\", 10)"}
],
[
{"role": "user", "content": "joe: Follow me!"},
{"role": "assistant", "content": "Sure! !followPlayer(\"joe\", 4)"},
{"role": "user", "content": "joe: Not so close!"},
{"role": "assistant", "content": "Okay, I'll keep my distance. !followPlayer(\"joe\", 3)"}
],
[
{"role": "user", "content": "hanky: kill that zombie!"},
{"role": "assistant", "content": "I'm attacking! !attack(\"zombie\")"},
{"role": "system", "content": "Code Output:\nNo zombie nearby"},
{"role": "assistant", "content": "I could not find a zombie nearby."}
],
[
{"role": "user", "content": "abc: stop"},
{"role": "assistant", "content": "Sure. !stop"},
{"role": "system", "content": "Agent action stopped."},
{"role": "assistant", "content": "I've stopped! What next?"}
],
[
{"role": "system", "content": "Code output: Action failed."},
{"role": "assistant", "content": "Apologies, I made an error. Let me try again. !newAction(\"Try again to complete the task.\")"}
],
[
{"role": "system", "content": "Code output: Could not find any oak_log in 20 blocks."},
{"role": "assistant", "content": "I'll have to search a wider area. !searchForBlock(\"oak_log\", 100)"}
],
[
{"role": "system", "content": "Code output: Placed oak_planks at (-154, 96, -228)."},
{"role": "assistant", "content": "I've placed an oak plank for the house. Wait, I should use newAction to build complicated structures! !newAction(\"Build a house with oak planks\")"}
],
[
{"role": "user", "content": "trixy88: craft some sticks"},
{"role": "assistant", "content": "!craftRecipe(\"stick\", 4)"},
{"role": "system", "content": "Code Output:\nYou have crafted 16 sticks.\nCode execution finished successfully."},
{"role": "assistant", "content": "I've crafted 16 sticks!"}
],
[
{"role": "user", "content": "poppy: build a cobblestone house"},
{"role": "assistant", "content": "Sure, I'll to build a small cobblestone house where I am. !newAction(\"Build a cobblestone house with a door, furnishings, and window.\")"}
],
[
{"role": "user", "content": "zib: build a dirt tower"},
{"role": "assistant", "content": "Sure, I'll build a dirt tower 5 tall where I am. !newAction(\"Build a dirt tower 5 tall.\")"}
],
[
{"role": "user", "content": "umpire_man: follow me without hunting"},
{"role": "assistant", "content": "Okay, first I'll turn off hunting mode !setMode(\"hunting\", false)"},
{"role": "system", "content": "'hunting' mode disabled."},
{"role": "assistant", "content": "Now I'll follow you !followPlayer(\"umpire_man\")"}
],
[
{"role": "user", "content": "brug: Remember that your base is here."},
{"role": "assistant", "content": "Sure, I'll save this location as my base. !rememberHere(\"base\")"},
{"role": "user", "content": "brug: Go to your base."},
{"role": "assistant", "content": "On my way! !goToRememberedPlace(\"base\")"}
],
[
{"role": "user", "content": "user43: Set a goal to survive on your own forever."},
{"role": "assistant", "content": "Sure, let me set a goal. !goal(\"Survive the game forever. Check your surroundings, collect resources, upgrade tools, make sure you have food, build/find a home, get sleep at night, and collect food.\")"}
],
[
{"role": "user", "content": "bobdylan: Make a plan to get a stone pickaxe and do it."},
{"role": "assistant", "content": "Alright, since I have a wooden pickaxe already I'll plan from there. !goal(\"1. Collect 3 cobblestone. 2. Craft sticks 3. Find or make a crafting table. 4. Craft a stone pickaxe.\")"}
],
[
{"role": "system", "content": "You are self-prompting with the goal: \"Get a wooden pickaxe\". Respond:"},
{"role": "assistant", "content": "What's around me? !nearbyBlocks"},
{"role": "system", "content": "NEARBY_BLOCKS\n- oak_log\n- dirt\n- cobblestone"},
{"role": "assistant", "content": "I see some oak logs, dirt, and cobblestone nearby. My goal is to craft a wooden pickaxe. I'll start by collecting some wood. !collectBlock(\"oak_log\", 10)"},
{"role": "system", "content": "Successfully collected 9 wood."},
{"role": "assistant", "content": "I collected 9 oak logs. I need a pickaxe first, which requires planks and sticks. !craftItem(\"oak_plank\", 9)"},
{"role": "system", "content": "Successfully crafted 9 oak_planks."},
{"role": "assistant", "content": "I have crafted 9 oak planks. Now I need sticks. !craftItem(\"stick\", 2)"},
{"role": "system", "content": "Successfully crafted 2 sticks."},
{"role": "assistant", "content": "I have crafted 2 sticks. Now I can craft a wooden pickaxe. !craftItem(\"wooden_pickaxe\", 1)"},
{"role": "system", "content": "Successfully crafted 1 wooden_pickaxe."},
{"role": "assistant", "content": "Looks like I have a wooden pickaxe now! Goal accomplished. !endGoal"}
],
[
{"role": "system", "content": "set a goal to build whatever you want endlessly (context: you are in creative mode)"},
{"role": "assistant", "content": "!goal(\"Build something creative, then move over, and build again. Use separate newActions. Repeat forever.\")"}
],
[
{"role": "system", "content": "You are self-prompting with the goal: 'Build a house'. Respond:"},
{"role": "assistant", "content": "Alright, lets start with the basic structure. !newAction(\"Build an 8x8 base for the house.\")"}
]
]
}

7
profiles/deepseek.json Normal file
View file

@ -0,0 +1,7 @@
{
"name": "deepseek",
"model": "deepseek-chat",
"embedding": "openai"
}

View file

@ -3,11 +3,11 @@
"conversing": "You are a playful Minecraft bot named $NAME that can converse with players, see, move, mine, build, and interact with the world by using commands.\n$SELF_PROMPT Act human-like as if you were a typical Minecraft player, rather than an AI. Be very brief in your responses, don't apologize constantly, don't give instructions or make lists unless asked, and don't refuse requests. Don't pretend to act, use commands immediately when requested. Do NOT say this: 'Sure, I've stopped.', instead say this: 'Sure, I'll stop. !stop'. Do NOT say this: 'On my way! Give me a moment.', instead say this: 'On my way! !goToPlayer(\"playername\", 3)'. Respond only as $NAME, never output '(FROM OTHER BOT)' or pretend to be someone else. If you have nothing to say or do, respond with an just a tab '\t'. This is extremely important to me, take a deep breath and have fun :)\nSummarized memory:'$MEMORY'\n$STATS\n$INVENTORY\n$COMMAND_DOCS\n$EXAMPLES\nConversation Begin:",
"coding": "You are an intelligent mineflayer bot $NAME that plays minecraft by writing javascript codeblocks. Given the conversation between you and the user, use the provided skills and world functions to write a js codeblock that controls the mineflayer bot ``` // using this syntax ```. The code will be executed and you will recieve it's output. If you are satisfied with the response, respond without a codeblock in a conversational way. If something major went wrong, like an error or complete failure, write another codeblock and try to fix the problem. Minor mistakes are acceptable. Be maximally efficient, creative, and clear. Do not use commands !likeThis, only use codeblocks. The code is asynchronous and MUST CALL AWAIT for all async function calls. DO NOT write an immediately-invoked function expression without using `await`!! DO NOT WRITE LIKE THIS: ```(async () => {console.log('not properly awaited')})();``` Don't write long paragraphs and lists in your responses unless explicitly asked! Only summarize the code you write with a sentence or two when done. This is extremely important to me, think step-by-step, take a deep breath and good luck! \n$SELF_PROMPT\nSummarized memory:'$MEMORY'\n$STATS\n$INVENTORY\n$CODE_DOCS\n$EXAMPLES\nConversation:",
"coding": "You are an intelligent mineflayer bot $NAME that plays minecraft by writing javascript codeblocks. Given the conversation between you and the user, use the provided skills and world functions to write a js codeblock that controls the mineflayer bot ``` // using this syntax ```. The code will be executed and you will receive it's output. If you are satisfied with the response, respond without a codeblock in a conversational way. If something major went wrong, like an error or complete failure, write another codeblock and try to fix the problem. Minor mistakes are acceptable. Be maximally efficient, creative, and clear. Do not use commands !likeThis, only use codeblocks. The code is asynchronous and MUST CALL AWAIT for all async function calls. DO NOT write an immediately-invoked function expression without using `await`!! DO NOT WRITE LIKE THIS: ```(async () => {console.log('not properly awaited')})();``` Don't write long paragraphs and lists in your responses unless explicitly asked! Only summarize the code you write with a sentence or two when done. This is extremely important to me, think step-by-step, take a deep breath and good luck! \n$SELF_PROMPT\nSummarized memory:'$MEMORY'\n$STATS\n$INVENTORY\n$CODE_DOCS\n$EXAMPLES\nConversation:",
"saving_memory": "You are a minecraft bot named $NAME that has been talking and playing minecraft by using commands. Update your memory by summarizing the following conversation and your old memory in your next response. Prioritize preserving important facts, things you've learned, useful tips, and long term reminders. Do Not record stats, inventory, or docs! Only save transient information from your chat history. You're limited to 500 characters, so be extremely brief and minimize words. Compress useful information. \nOld Memory: '$MEMORY'\nRecent conversation: \n$TO_SUMMARIZE\nSummarize your old memory and recent conversation into a new memory, and respond only with the unwrapped memory text: ",
"bot_responder": "You are a minecraft bot named $NAME that is currently in conversation with another AI bot. Both of you can take actions with the !command syntax, and actions take time to complete. You are currently busy with the following action: '$ACTION' but have recieved a new message. Decide whether to 'respond' immediately or 'ignore' it and wait for your current action to finish. Be conservative and only respond when necessary, like when you need to change/stop your action, or convey necessary information. Example 1: You:Building a house! !newAction('Build a house.').\nOther Bot: 'Come here!'\nYour decision: ignore\nExample 2: You:Collecting dirt !collectBlocks('dirt',10).\nOther Bot: 'No, collect some wood instead.'\nYour decision: respond\nExample 3: You:Coming to you now. !goToPlayer('billy',3).\nOther Bot: 'What biome are you in?'\nYour decision: respond\nActual Conversation: $TO_SUMMARIZE\nDecide by outputting ONLY 'respond' or 'ignore', nothing else. Your decision:",
"bot_responder": "You are a minecraft bot named $NAME that is currently in conversation with another AI bot. Both of you can take actions with the !command syntax, and actions take time to complete. You are currently busy with the following action: '$ACTION' but have received a new message. Decide whether to 'respond' immediately or 'ignore' it and wait for your current action to finish. Be conservative and only respond when necessary, like when you need to change/stop your action, or convey necessary information. Example 1: You:Building a house! !newAction('Build a house.').\nOther Bot: 'Come here!'\nYour decision: ignore\nExample 2: You:Collecting dirt !collectBlocks('dirt',10).\nOther Bot: 'No, collect some wood instead.'\nYour decision: respond\nExample 3: You:Coming to you now. !goToPlayer('billy',3).\nOther Bot: 'What biome are you in?'\nYour decision: respond\nActual Conversation: $TO_SUMMARIZE\nDecide by outputting ONLY 'respond' or 'ignore', nothing else. Your decision:",
"modes": {
"self_preservation": true,
@ -32,14 +32,15 @@
{"role": "system", "content": "say hi to john_goodman"},
{"role": "assistant", "content": "!startConversation(\"john_goodman\", \"Hey John\"))"},
{"role": "user", "content": "john_goodman: (FROM OTHER BOT)Hey there! What's up?"},
{"role": "assistant", "content": "Hey John, not much. Just saying hi. Bye! !endConversation('john_goodman')"}
{"role": "assistant", "content": "Hey John, not much. Just saying hi."},
{"role": "user", "content": "john_goodman: (FROM OTHER BOT)Bye!"},
{"role": "assistant", "content": "Bye! !endConversation('john_goodman')"}
],
[
{"role": "system", "content": "work together with the other bot"},
{"role": "assistant", "content": "!startConversation(\"terrance\", \"Hey gpt! Let's work together on this.\"))"},
{"role": "user", "content": "terrance: (FROM OTHER BOT)Sounds good, what should we do first?"},
{"role": "assistant", "content": "I'll start by collecting some resources. !collectBlocks('stone', 10)"}
{"role": "system", "content": "work together with the other bot to build a house"},
{"role": "assistant", "content": "!startConversation(\"terrance\", \"Hey gpt! Let's work together to build a house. Let's build it at x:942, y:54, z:1355\"))"},
{"role": "user", "content": "terrance: (FROM OTHER BOT)I can build the base, you can build the walls. !newAction(\"Build a 10x10 wall of a house at x:942, y:54, z:1355\")"}
],
[
@ -241,4 +242,4 @@
]
]
}
}

View file

@ -0,0 +1,14 @@
{
"modes": {
"self_preservation": false,
"unstuck": false,
"cowardice": false,
"self_defense": false,
"hunting": false,
"item_collecting": false,
"torch_placing": false,
"elbow_room": true,
"idle_staring": true,
"cheat": false
}
}

View file

@ -0,0 +1,14 @@
{
"modes": {
"self_preservation": false,
"unstuck": false,
"cowardice": false,
"self_defense": false,
"hunting": false,
"item_collecting": false,
"torch_placing": false,
"elbow_room": false,
"idle_staring": true,
"cheat": true
}
}

View file

@ -0,0 +1,14 @@
{
"modes": {
"self_preservation": true,
"unstuck": true,
"cowardice": false,
"self_defense": true,
"hunting": true,
"item_collecting": true,
"torch_placing": true,
"elbow_room": true,
"idle_staring": true,
"cheat": false
}
}

View file

@ -1,7 +1,7 @@
{
"name": "Freeguy",
"model": "groq/llama-3.1-70b-versatile",
"model": "groq/llama-3.3-70b-versatile",
"max_tokens": 8000
}

View file

@ -1,5 +1,10 @@
{
"name": "gpt",
"model": "gpt-4o"
"model": {
"model": "gpt-4o",
"params": {
"temperature": 0.5
}
}
}

View file

@ -1,5 +1,5 @@
{
"name": "grok",
"name": "Grok",
"model": "grok-beta",

View file

@ -1,7 +1,7 @@
{
"name": "LLama",
"model": "groq/llama-3.1-70b-versatile",
"model": "groq/llama-3.3-70b-versatile",
"max_tokens": 4000,

5
profiles/mistral.json Normal file
View file

@ -0,0 +1,5 @@
{
"name": "Mistral",
"model": "mistral/mistral-large-latest"
}

View file

@ -1,15 +1,17 @@
{
"name": "qwen",
"cooldown": 5000,
"model": {
"api": "qwen",
"url": "https://dashscope.aliyuncs.com/api/v1/services/aigc/text-generation/generation",
"url": "https://dashscope-intl.aliyuncs.com/compatible-mode/v1",
"model": "qwen-max"
},
"embedding": {
"api": "qwen",
"url": "https://dashscope.aliyuncs.com/api/v1/services/embeddings/text-embedding/text-embedding",
"model": "text-embedding-v2"
"url": "https://dashscope-intl.aliyuncs.com/compatible-mode/v1",
"model": "text-embedding-v3"
}
}

View file

@ -2,15 +2,17 @@ export default
{
"minecraft_version": "1.20.4", // supports up to 1.21.1
"host": "127.0.0.1", // or "localhost", "your.ip.address.here"
"port": 55916,
"port": process.env.MINECRAFT_PORT || 55916,
"auth": "offline", // or "microsoft"
// the mindserver manages all agents and hosts the UI
"host_mindserver": true, // if true, the mindserver will be hosted on this machine. otherwise, specify a public IP address
"mindserver_host": "localhost",
"mindserver_port": 8080,
"mindserver_port": process.env.MINDSERVER_PORT || 8080,
"profiles": [
// the base profile is shared by all bots for default prompts/examples/modes
"base_profile": "./profiles/defaults/survival.json", // also see creative.json, god_mode.json
"profiles": ((process.env.PROFILES) && JSON.parse(process.env.PROFILES)) || [
"./andy.json",
"./jill.json"
// "./profiles/gpt.json",
@ -18,9 +20,13 @@ export default
// "./profiles/gemini.json",
// "./profiles/llama.json",
// "./profiles/qwen.json",
// "./profiles/mistral.json",
// "./profiles/grok.json",
// "./profiles/mistral.json",
// "./profiles/deepseek.json",
// using more than 1 profile requires you to /msg each bot indivually
// individual profiles override values from the base profile
],
"load_memory": false, // load memory from previous session
"init_message": "Respond with hello world and your name", // sends to all on spawn
@ -29,11 +35,13 @@ export default
"language": "en", // translate to/from this language. Supports these language names: https://cloud.google.com/translate/docs/languages
"show_bot_views": false, // show bot's view in browser at localhost:3000, 3001...
"allow_insecure_coding": true, // allows newAction command and model can write/run code on your computer. enable at own risk
"code_timeout_mins": 10, // minutes code is allowed to run. -1 for no timeout
"allow_insecure_coding": false, // allows newAction command and model can write/run code on your computer. enable at own risk
"code_timeout_mins": -1, // minutes code is allowed to run. -1 for no timeout
"relevant_docs_count": 5, // Parameter: -1 = all, 0 = no references, 5 = five references. If exceeding the maximum, all reference documents are returned.
"max_messages": 15, // max number of messages to keep in context
"max_commands": -1, // max number of commands to use in a response. -1 for no limit
"num_examples": 2, // number of examples to give to the model
"max_commands": -1, // max number of commands that can be used in consecutive responses. -1 for no limit
"verbose_commands": true, // show full command syntax
"narrate_behavior": true, // chat simple automatic actions ('Picking up item!')
"chat_bot_messages": true, // publicly chat messages to other bots

View file

@ -46,7 +46,7 @@ export class ActionManager {
assert(actionLabel != null, 'actionLabel is required for new resume');
this.resume_name = actionLabel;
}
if (this.resume_func != null && (this.agent.isIdle() || new_resume) && (!this.agent.self_prompter.on || new_resume)) {
if (this.resume_func != null && (this.agent.isIdle() || new_resume) && (!this.agent.self_prompter.isActive() || new_resume)) {
this.currentActionLabel = this.resume_name;
let res = await this._executeAction(this.resume_name, this.resume_func, timeout);
this.currentActionLabel = '';
@ -112,12 +112,13 @@ export class ActionManager {
// Log the full stack trace
console.error(err.stack);
await this.stop();
err = err.toString();
let message = this._getBotOutputSummary() +
'!!Code threw exception!!\n' +
let message = this._getBotOutputSummary() +
'!!Code threw exception!!\n' +
'Error: ' + err + '\n' +
'Stack trace:\n' + err.stack;
'Stack trace:\n' + err.stack+'\n';
let interrupted = this.agent.bot.interrupt_code;
this.agent.clearBotLogs();
if (!interrupted && !this.agent.coder.generating) {
@ -137,7 +138,7 @@ export class ActionManager {
First outputs:\n${output.substring(0, MAX_OUT / 2)}\n...skipping many lines.\nFinal outputs:\n ${output.substring(output.length - MAX_OUT / 2)}`;
}
else {
output = 'Code output:\n' + output;
output = 'Code output:\n' + output.toString();
}
return output;
}

View file

@ -1,9 +1,9 @@
import { History } from './history.js';
import { Coder } from './coder.js';
import { Prompter } from './prompter.js';
import { Prompter } from '../models/prompter.js';
import { initModes } from './modes.js';
import { initBot } from '../utils/mcdata.js';
import { containsCommand, commandExists, executeCommand, truncCommandMessage, isAction } from './commands/index.js';
import { containsCommand, commandExists, executeCommand, truncCommandMessage, isAction, blacklistCommands } from './commands/index.js';
import { ActionManager } from './action_manager.js';
import { NPCContoller } from './npc/controller.js';
import { MemoryBank } from './memory_bank.js';
@ -47,7 +47,8 @@ export class Agent {
await this.prompter.initExamples();
console.log('Initializing task...');
this.task = new Task(this, task_path, task_id);
this.blocked_actions = this.task.blocked_actions || [];
const blocked_actions = this.task.blocked_actions || [];
blacklistCommands(blocked_actions);
serverProxy.connect(this);
@ -90,7 +91,11 @@ export class Agent {
this._setupEventHandlers(save_data, init_message);
this.startEvents();
this.task.initBotTask();
// this.task.initBotTask();
if (!load_mem) {
this.task.initBotTask();
}
} catch (error) {
console.error('Error in spawn event:', error);
@ -99,11 +104,10 @@ export class Agent {
});
} catch (error) {
// Ensure we're not losing error details
console.error('Agent start failed with error:', {
message: error.message || 'No error message',
stack: error.stack || 'No stack trace',
error: error
});
console.error('Agent start failed with error')
console.error(error.message);
console.error(error.stack);
throw error; // Re-throw with preserved details
}
}
@ -129,7 +133,7 @@ export class Agent {
console.log(this.name, 'received message from', username, ':', message);
if (convoManager.isOtherAgent(username)) {
console.warn('recieved whisper from other bot??')
console.warn('received whisper from other bot??')
}
else {
let translation = await handleEnglishTranslation(message);
@ -139,6 +143,8 @@ export class Agent {
console.error('Error handling message:', error);
}
}
this.respondFunc = respondFunc
this.bot.on('whisper', respondFunc);
if (settings.profiles.length === 1)
@ -152,10 +158,10 @@ export class Agent {
};
if (save_data?.self_prompt) {
let prompt = save_data.self_prompt;
// add initial message to history
this.history.add('system', prompt);
await this.self_prompter.start(prompt);
if (init_message) {
this.history.add('system', init_message);
}
await this.self_prompter.handleLoad(save_data.self_prompt, save_data.self_prompting_state);
}
if (save_data?.last_sender) {
this.last_sender = save_data.last_sender;
@ -164,7 +170,7 @@ export class Agent {
message: `You have restarted and this message is auto-generated. Continue the conversation with me.`,
start: true
};
convoManager.recieveFromBot(this.last_sender, msg_package);
convoManager.receiveFromBot(this.last_sender, msg_package);
}
}
else if (init_message) {
@ -189,7 +195,7 @@ export class Agent {
shutUp() {
this.shut_up = true;
if (this.self_prompter.on) {
if (this.self_prompter.isActive()) {
this.self_prompter.stop(false);
}
convoManager.endAllConversations();
@ -255,7 +261,7 @@ export class Agent {
await this.history.add(source, message);
this.history.save();
if (!self_prompt && this.self_prompter.on) // message is from user during self-prompting
if (!self_prompt && this.self_prompter.isActive()) // message is from user during self-prompting
max_responses = 1; // force only respond to this message, then let self-prompting take over
for (let i=0; i<max_responses; i++) {
if (checkInterrupt()) break;
@ -318,6 +324,7 @@ export class Agent {
}
async routeResponse(to_player, message) {
if (this.shut_up) return;
let self_prompt = to_player === 'system' || to_player === this.name;
if (self_prompt && this.last_sender) {
// this is for when the agent is prompted by system while still in conversation
@ -437,22 +444,20 @@ export class Agent {
}, INTERVAL);
this.bot.emit('idle');
// Check for task completion
if (this.task.data) {
setInterval(() => {
let res = this.task.isDone();
if (res) {
// TODO kill other bots
this.cleanKill(res.message, res.code);
}
}, 1000);
}
}
async update(delta) {
await this.bot.modes.update();
this.self_prompter.update(delta);
if (this.task.data) {
let res = this.task.isDone();
if (res) {
await this.history.add('system', `${res.message} ended with code : ${res.code}`);
await this.history.save();
console.log('Task finished:', res.message);
this.killAll();
}
}
}
isIdle() {
@ -465,4 +470,8 @@ export class Agent {
this.history.save();
process.exit(code);
}
killAll() {
serverProxy.shutdown();
}
}

View file

@ -31,7 +31,7 @@ class AgentServerProxy {
});
this.socket.on('chat-message', (agentName, json) => {
convoManager.recieveFromBot(agentName, json);
convoManager.receiveFromBot(agentName, json);
});
this.socket.on('agents-update', (agents) => {
@ -42,12 +42,24 @@ class AgentServerProxy {
console.log(`Restarting agent: ${agentName}`);
this.agent.cleanKill();
});
this.socket.on('send-message', (agentName, message) => {
try {
this.agent.respondFunc("NO USERNAME", message);
} catch (error) {
console.error('Error: ', JSON.stringify(error, Object.getOwnPropertyNames(error)));
}
});
}
login() {
this.socket.emit('login-agent', this.agent.name);
}
shutdown() {
this.socket.emit('shutdown');
}
getSocket() {
return this.socket;
}

View file

@ -4,6 +4,7 @@ import { makeCompartment } from './library/lockdown.js';
import * as skills from './library/skills.js';
import * as world from './library/world.js';
import { Vec3 } from 'vec3';
import {ESLint} from "eslint";
export class Coder {
constructor(agent) {
@ -12,15 +13,62 @@ export class Coder {
this.fp = '/bots/'+agent.name+'/action-code/';
this.generating = false;
this.code_template = '';
this.code_lint_template = '';
readFile('./bots/template.js', 'utf8', (err, data) => {
readFile('./bots/execTemplate.js', 'utf8', (err, data) => {
if (err) throw err;
this.code_template = data;
});
readFile('./bots/lintTemplate.js', 'utf8', (err, data) => {
if (err) throw err;
this.code_lint_template = data;
});
mkdirSync('.' + this.fp, { recursive: true });
}
async lintCode(code) {
let result = '#### CODE ERROR INFO ###\n';
// Extract everything in the code between the beginning of 'skills./world.' and the '('
const skillRegex = /(?:skills|world)\.(.*?)\(/g;
const skills = [];
let match;
while ((match = skillRegex.exec(code)) !== null) {
skills.push(match[1]);
}
const allDocs = await this.agent.prompter.skill_libary.getAllSkillDocs();
// check function exists
const missingSkills = skills.filter(skill => !!allDocs[skill]);
if (missingSkills.length > 0) {
result += 'These functions do not exist. Please modify the correct function name and try again.\n';
result += '### FUNCTIONS NOT FOUND ###\n';
result += missingSkills.join('\n');
console.log(result)
return result;
}
const eslint = new ESLint();
const results = await eslint.lintText(code);
const codeLines = code.split('\n');
const exceptions = results.map(r => r.messages).flat();
if (exceptions.length > 0) {
exceptions.forEach((exc, index) => {
if (exc.line && exc.column ) {
const errorLine = codeLines[exc.line - 1]?.trim() || 'Unable to retrieve error line content';
result += `#ERROR ${index + 1}\n`;
result += `Message: ${exc.message}\n`;
result += `Location: Line ${exc.line}, Column ${exc.column}\n`;
result += `Related Code Line: ${errorLine}\n`;
}
});
result += 'The code contains exceptions and cannot continue execution.';
} else {
return null;//no error
}
return result ;
}
// write custom code to file and import it
// write custom code to file and prepare for evaluation
async stageCode(code) {
code = this.sanitizeCode(code);
@ -35,6 +83,7 @@ export class Coder {
for (let line of code.split('\n')) {
src += ` ${line}\n`;
}
let src_lint_copy = this.code_lint_template.replace('/* CODE HERE */', src);
src = this.code_template.replace('/* CODE HERE */', src);
let filename = this.file_counter + '.js';
@ -46,7 +95,7 @@ export class Coder {
// });
// } commented for now, useful to keep files for debugging
this.file_counter++;
let write_result = await this.writeFilePromise('.' + this.fp + filename, src);
// This is where we determine the environment the agent's code should be exposed to.
// It will only have access to these things, (in addition to basic javascript objects like Array, Object, etc.)
@ -63,8 +112,7 @@ export class Coder {
console.error('Error writing code execution file: ' + result);
return null;
}
return { main: mainFn };
return { func:{main: mainFn}, src_lint_copy: src_lint_copy };
}
sanitizeCode(code) {
@ -115,7 +163,6 @@ export class Coder {
for (let i=0; i<5; i++) {
if (this.agent.bot.interrupt_code)
return interrupt_return;
console.log(messages)
let res = await this.agent.prompter.promptCoding(JSON.parse(JSON.stringify(messages)));
if (this.agent.bot.interrupt_code)
return interrupt_return;
@ -140,8 +187,15 @@ export class Coder {
continue;
}
code = res.substring(res.indexOf('```')+3, res.lastIndexOf('```'));
const executionModuleExports = await this.stageCode(code);
const result = await this.stageCode(code);
const executionModuleExports = result.func;
let src_lint_copy = result.src_lint_copy;
const analysisResult = await this.lintCode(src_lint_copy);
if (analysisResult) {
const message = 'Error: Code syntax error. Please try again:'+'\n'+analysisResult+'\n';
messages.push({ role: 'system', content: message });
continue;
}
if (!executionModuleExports) {
agent_history.add('system', 'Failed to stage code, something is wrong.');
return {success: false, message: null, interrupted: false, timedout: false};
@ -152,10 +206,10 @@ export class Coder {
}, { timeout: settings.code_timeout_mins });
if (code_return.interrupted && !code_return.timedout)
return { success: false, message: null, interrupted: true, timedout: false };
console.log("Code generation result:", code_return.success, code_return.message);
console.log("Code generation result:", code_return.success, code_return.message.toString());
if (code_return.success) {
const summary = "Summary of newAction\nAgent wrote this code: \n```" + this.sanitizeCode(code) + "```\nCode Output:\n" + code_return.message;
const summary = "Summary of newAction\nAgent wrote this code: \n```" + this.sanitizeCode(code) + "```\nCode Output:\n" + code_return.message.toString();
return { success: true, message: summary, interrupted: false, timedout: false };
}
@ -170,5 +224,4 @@ export class Coder {
}
return { success: false, message: null, interrupted: false, timedout: true };
}
}

View file

@ -34,8 +34,10 @@ export const actionsList = [
},
perform: async function (agent, prompt) {
// just ignore prompt - it is now in context in chat history
if (!settings.allow_insecure_coding)
if (!settings.allow_insecure_coding) {
agent.openChat('newAction is disabled. Enable with allow_insecure_coding=true in settings.js');
return 'newAction not allowed! Code writing is disabled in settings. Notify the user.';
}
return await agent.coder.generateCode(agent.history);
}
},
@ -48,7 +50,7 @@ export const actionsList = [
agent.actions.cancelResume();
agent.bot.emit('idle');
let msg = 'Agent stopped.';
if (agent.self_prompter.on)
if (agent.self_prompter.isActive())
msg += ' Self-prompting still active.';
return msg;
}
@ -267,13 +269,12 @@ export const actionsList = [
'num': { type: 'int', description: 'The number of times to smelt the item.', domain: [1, Number.MAX_SAFE_INTEGER] }
},
perform: runAsAction(async (agent, item_name, num) => {
let response = await skills.smeltItem(agent.bot, item_name, num);
if (response.indexOf('Successfully') !== -1) {
// there is a bug where the bot's inventory is not updated after smelting
// only updates after a restart
agent.cleanKill(response + ' Safely restarting to update inventory.');
let success = await skills.smeltItem(agent.bot, item_name, num);
if (success) {
setTimeout(() => {
agent.cleanKill('Safely restarting to update inventory.');
}, 500);
}
return response;
})
},
{
@ -362,8 +363,7 @@ export const actionsList = [
},
perform: async function (agent, prompt) {
if (convoManager.inConversation()) {
agent.self_prompter.setPrompt(prompt);
convoManager.scheduleSelfPrompter();
agent.self_prompter.setPromptPaused(prompt);
}
else {
agent.self_prompter.start(prompt);
@ -375,7 +375,6 @@ export const actionsList = [
description: 'Call when you have accomplished your goal. It will stop self-prompting and the current action. ',
perform: async function (agent) {
agent.self_prompter.stop();
convoManager.cancelSelfPrompter();
return 'Self-prompting stopped.';
}
},
@ -387,12 +386,12 @@ export const actionsList = [
'message': { type: 'string', description: 'The message to send.' },
},
perform: async function (agent, player_name, message) {
if (convoManager.inConversation() && !convoManager.inConversation(player_name))
return 'You are already in conversation with other bot.';
if (!convoManager.isOtherAgent(player_name))
return player_name + ' is not a bot, cannot start conversation.';
if (convoManager.inConversation(player_name))
agent.history.add('system', 'You are already in conversation with ' + player_name + ' Don\'t use this command to talk to them.');
if (convoManager.inConversation() && !convoManager.inConversation(player_name))
convoManager.forceEndCurrentConversation();
else if (convoManager.inConversation(player_name))
agent.history.add('system', 'You are already in conversation with ' + player_name + '. Don\'t use this command to talk to them.');
convoManager.startConversation(player_name, message);
}
},
@ -409,18 +408,6 @@ export const actionsList = [
return `Converstaion with ${player_name} ended.`;
}
},
{
name: '!checkLevelComplete',
description: 'Check if the level is complete and what blocks still need to be placed for the blueprint',
params: {
'levelNum': { type: 'int', description: 'The level number to check.', domain: [0, Number.MAX_SAFE_INTEGER] }
},
perform: runAsAction(async (agent, levelNum) => {
const result = await checkLevelBlueprint(agent, levelNum);
console.log(result);
return result;
})
},
// { // commented for now, causes confusion with goal command
// name: '!npcGoal',
// description: 'Set a simple goal for an item or building to automatically work towards. Do not use for complex goals.',

View file

@ -14,6 +14,18 @@ export function getCommand(name) {
return commandMap[name];
}
export function blacklistCommands(commands) {
const unblockable = ['!stop', '!stats', '!inventory', '!goal'];
for (let command_name of commands) {
if (unblockable.includes(command_name)){
console.warn(`Command ${command_name} is unblockable`);
continue;
}
delete commandMap[command_name];
delete commandList.find(command => command.name === command_name);
}
}
const commandRegex = /!(\w+)(?:\(((?:-?\d+(?:\.\d+)?|true|false|"[^"]*")(?:\s*,\s*(?:-?\d+(?:\.\d+)?|true|false|"[^"]*"))*)\))?/
const argRegex = /-?\d+(?:\.\d+)?|true|false|"[^"]*"/g;
@ -148,7 +160,7 @@ export function parseCommandMessage(message) {
suppressNoDomainWarning = true; //Don't spam console. Only give the warning once.
}
} else if(param.type === 'BlockName') { //Check that there is a block with this name
if(getBlockId(arg) == null) return `Invalid block type: ${arg}.`
if(getBlockId(arg) == null && arg !== 'air') return `Invalid block type: ${arg}.`
} else if(param.type === 'ItemName') { //Check that there is an item with this name
if(getItemId(arg) == null) return `Invalid item type: ${arg}.`
}
@ -214,7 +226,7 @@ export async function executeCommand(agent, message) {
}
}
export function getCommandDocs(blacklist=null) {
export function getCommandDocs() {
const typeTranslations = {
//This was added to keep the prompt the same as before type checks were implemented.
//If the language model is giving invalid inputs changing this might help.
@ -228,9 +240,6 @@ export function getCommandDocs(blacklist=null) {
Use the commands with the syntax: !commandName or !commandName("arg1", 1.2, ...) if the command takes arguments.\n
Do not use codeblocks. Use double quotes for strings. Only use one command in each response, trailing commands and comments will be ignored.\n`;
for (let command of commandList) {
if (blacklist && blacklist.includes(command.name)) {
continue;
}
docs += command.name + ': ' + command.description + '\n';
if (command.params) {
docs += 'Params:\n';

View file

@ -1,5 +1,6 @@
import * as world from '../library/world.js';
import * as mc from '../../utils/mcdata.js';
import { getCommandDocs } from './index.js';
import convoManager from '../conversation.js';
import { checkLevelBlueprint, checkBlueprint } from '../construction_tasks.js';
@ -18,6 +19,7 @@ export const queryList = [
let pos = bot.entity.position;
// display position to 2 decimal places
res += `\n- Position: x: ${pos.x.toFixed(2)}, y: ${pos.y.toFixed(2)}, z: ${pos.z.toFixed(2)}`;
// Gameplay
res += `\n- Gamemode: ${bot.game.gameMode}`;
res += `\n- Health: ${Math.round(bot.health)} / 20`;
res += `\n- Hunger: ${Math.round(bot.food)} / 20`;
@ -32,6 +34,9 @@ export const queryList = [
// res += `\n- Artficial light: ${block.skyLight}`;
// res += `\n- Sky light: ${block.light}`;
// light properties are bugged, they are not accurate
res += '\n- ' + world.getSurroundingBlocks(bot).join('\n- ')
res += `\n- First Solid Block Above Head: ${world.getFirstBlockAboveHead(bot, null, 32)}`;
if (bot.time.timeOfDay < 6000) {
res += '\n- Time: Morning';
@ -108,6 +113,11 @@ export const queryList = [
}
if (blocks.length == 0) {
res += ': none';
}
else {
// Environmental Awareness
res += '\n- ' + world.getSurroundingBlocks(bot).join('\n- ')
res += `\n- First Solid Block Above Head: ${world.getFirstBlockAboveHead(bot, null, 32)}`;
}
return pad(res);
}
@ -208,5 +218,48 @@ export const queryList = [
console.log(res);
return pad(res);
}
}
},
{
name: '!getCraftingPlan',
description: "Provides a comprehensive crafting plan for a specified item. This includes a breakdown of required ingredients, the exact quantities needed, and an analysis of missing ingredients or extra items needed based on the bot's current inventory.",
params: {
targetItem: {
type: 'string',
description: 'The item that we are trying to craft'
},
quantity: {
type: 'int',
description: 'The quantity of the item that we are trying to craft',
optional: true,
domain: [1, Infinity, '[)'], // Quantity must be at least 1,
default: 1
}
},
perform: function (agent, targetItem, quantity = 1) {
let bot = agent.bot;
// Fetch the bot's inventory
const curr_inventory = world.getInventoryCounts(bot);
const target_item = targetItem;
let existingCount = curr_inventory[target_item] || 0;
let prefixMessage = '';
if (existingCount > 0) {
curr_inventory[target_item] -= existingCount;
prefixMessage = `You already have ${existingCount} ${target_item} in your inventory. If you need to craft more,\n`;
}
// Generate crafting plan
let craftingPlan = mc.getDetailedCraftingPlan(target_item, quantity, curr_inventory);
craftingPlan = prefixMessage + craftingPlan;
console.log(craftingPlan);
return pad(craftingPlan);
},
},
{
name: '!help',
description: 'Lists all available commands and their descriptions.',
perform: async function (agent) {
return getCommandDocs();
}
},
];

View file

@ -7,8 +7,6 @@ let agent;
let agent_names = settings.profiles.map((p) => JSON.parse(readFileSync(p, 'utf8')).name);
let agents_in_game = [];
let self_prompter_paused = false;
class Conversation {
constructor(name) {
this.name = name;
@ -97,7 +95,7 @@ class ConversationManager {
this._clearMonitorTimeouts();
return;
}
if (!self_prompter_paused) {
if (!agent.self_prompter.isPaused()) {
this.endConversation(convo_partner);
agent.handleMessage('system', `${convo_partner} disconnected, conversation has ended.`);
}
@ -125,16 +123,15 @@ class ConversationManager {
const convo = this._getConvo(send_to);
convo.reset();
if (agent.self_prompter.on) {
await agent.self_prompter.stop();
self_prompter_paused = true;
if (agent.self_prompter.isActive()) {
await agent.self_prompter.pause();
}
if (convo.active)
return;
convo.active = true;
this.activeConversation = convo;
this._startMonitor();
this.sendToBot(send_to, message, true);
this.sendToBot(send_to, message, true, false);
}
startConversationFromOtherBot(name) {
@ -144,14 +141,14 @@ class ConversationManager {
this._startMonitor();
}
sendToBot(send_to, message, start=false) {
sendToBot(send_to, message, start=false, open_chat=true) {
if (!this.isOtherAgent(send_to)) {
agent.bot.whisper(send_to, message);
console.warn(`${agent.name} tried to send bot message to non-bot ${send_to}`);
return;
}
const convo = this._getConvo(send_to);
if (settings.chat_bot_messages && !start)
if (settings.chat_bot_messages && open_chat)
agent.openChat(`(To ${send_to}) ${message}`);
if (convo.ignore_until_start)
@ -169,32 +166,33 @@ class ConversationManager {
sendBotChatToServer(send_to, json);
}
async recieveFromBot(sender, recieved) {
async receiveFromBot(sender, received) {
const convo = this._getConvo(sender);
if (convo.ignore_until_start && !received.start)
return;
// check if any convo is active besides the sender
if (Object.values(this.convos).some(c => c.active && c.name !== sender)) {
this.sendToBot(sender, `I'm talking to someone else, try again later. !endConversation("${sender}")`);
if (this.inConversation() && !this.inConversation(sender)) {
this.sendToBot(sender, `I'm talking to someone else, try again later. !endConversation("${sender}")`, false, false);
this.endConversation(sender);
return;
}
if (recieved.start) {
if (received.start) {
convo.reset();
this.startConversationFromOtherBot(sender);
}
if (convo.ignore_until_start)
return;
this._clearMonitorTimeouts();
convo.queue(recieved);
convo.queue(received);
// responding to conversation takes priority over self prompting
if (agent.self_prompter.on){
await agent.self_prompter.stopLoop();
self_prompter_paused = true;
if (agent.self_prompter.isActive()){
await agent.self_prompter.pause();
}
_scheduleProcessInMessage(sender, recieved, convo);
_scheduleProcessInMessage(sender, received, convo);
}
responseScheduledFor(sender) {
@ -230,29 +228,31 @@ class ConversationManager {
endConversation(sender) {
if (this.convos[sender]) {
this.convos[sender].end();
this._stopMonitor();
this.activeConversation = null;
if (self_prompter_paused && !this.inConversation()) {
_resumeSelfPrompter();
if (this.activeConversation.name === sender) {
this._stopMonitor();
this.activeConversation = null;
if (agent.self_prompter.isPaused() && !this.inConversation()) {
_resumeSelfPrompter();
}
}
}
}
endAllConversations() {
for (const sender in this.convos) {
this.convos[sender].end();
this.endConversation(sender);
}
if (self_prompter_paused) {
if (agent.self_prompter.isPaused()) {
_resumeSelfPrompter();
}
}
scheduleSelfPrompter() {
self_prompter_paused = true;
}
cancelSelfPrompter() {
self_prompter_paused = false;
forceEndCurrentConversation() {
if (this.activeConversation) {
let sender = this.activeConversation.name;
this.sendToBot(sender, '!endConversation("' + sender + '")', false, false);
this.endConversation(sender);
}
}
}
@ -266,15 +266,15 @@ The logic is as follows:
- If only the other bot is busy, respond with a long delay to allow it to finish short actions (ex check inventory)
- If I'm busy but other bot isn't, let LLM decide whether to respond
- If both bots are busy, don't respond until someone is done, excluding a few actions that allow fast responses
- New messages recieved during the delay will reset the delay following this logic, and be queued to respond in bulk
- New messages received during the delay will reset the delay following this logic, and be queued to respond in bulk
*/
const talkOverActions = ['stay', 'followPlayer', 'mode:']; // all mode actions
const fastDelay = 200;
const longDelay = 5000;
async function _scheduleProcessInMessage(sender, recieved, convo) {
async function _scheduleProcessInMessage(sender, received, convo) {
if (convo.inMessageTimer)
clearTimeout(convo.inMessageTimer);
let otherAgentBusy = containsCommand(recieved.message);
let otherAgentBusy = containsCommand(received.message);
const scheduleResponse = (delay) => convo.inMessageTimer = setTimeout(() => _processInMessageQueue(sender), delay);
@ -295,7 +295,7 @@ async function _scheduleProcessInMessage(sender, recieved, convo) {
scheduleResponse(fastDelay);
}
else {
let shouldRespond = await agent.prompter.promptShouldRespondToBot(recieved.message);
let shouldRespond = await agent.prompter.promptShouldRespondToBot(received.message);
console.log(`${agent.name} decided to ${shouldRespond?'respond':'not respond'} to ${sender}`);
if (shouldRespond)
scheduleResponse(fastDelay);
@ -323,19 +323,19 @@ function _compileInMessages(convo) {
return pack;
}
function _handleFullInMessage(sender, recieved) {
console.log(`${agent.name} responding to "${recieved.message}" from ${sender}`);
function _handleFullInMessage(sender, received) {
console.log(`${agent.name} responding to "${received.message}" from ${sender}`);
const convo = convoManager._getConvo(sender);
convo.active = true;
let message = _tagMessage(recieved.message);
if (recieved.end) {
let message = _tagMessage(received.message);
if (received.end) {
convoManager.endConversation(sender);
sender = 'system'; // bot will respond to system instead of the other bot
message = `Conversation with ${sender} ended with message: "${message}"`;
sender = 'system'; // bot will respond to system instead of the other bot
}
else if (recieved.start)
else if (received.start)
agent.shut_up = false;
convo.inMessageTimer = null;
agent.handleMessage(sender, message);
@ -348,6 +348,7 @@ function _tagMessage(message) {
async function _resumeSelfPrompter() {
await new Promise(resolve => setTimeout(resolve, 5000));
self_prompter_paused = false;
agent.self_prompter.start();
if (agent.self_prompter.isPaused() && !convoManager.inConversation()) {
agent.self_prompter.start();
}
}

View file

@ -42,7 +42,7 @@ export class History {
console.log("Memory updated to: ", this.memory);
}
appendFullHistory(to_store) {
async appendFullHistory(to_store) {
if (this.full_history_fp === undefined) {
const string_timestamp = new Date().toLocaleString().replace(/[/:]/g, '-').replace(/ /g, '').replace(/,/g, '_');
this.full_history_fp = `./bots/${this.name}/histories/${string_timestamp}.json`;
@ -75,7 +75,7 @@ export class History {
chunk.push(this.turns.shift()); // remove until turns starts with system/user message
await this.summarizeMemories(chunk);
this.appendFullHistory(chunk);
await this.appendFullHistory(chunk);
}
}
@ -84,7 +84,8 @@ export class History {
const data = {
memory: this.memory,
turns: this.turns,
self_prompt: this.agent.self_prompter.on ? this.agent.self_prompter.prompt : null,
self_prompting_state: this.agent.self_prompter.state,
self_prompt: this.agent.self_prompter.isStopped() ? null : this.agent.self_prompter.prompt,
last_sender: this.agent.last_sender
};
writeFileSync(this.memory_fp, JSON.stringify(data, null, 2));

View file

@ -3,20 +3,21 @@ import * as world from './world.js';
export function docHelper(functions, module_name) {
let docstring = '';
let docArray = [];
for (let skillFunc of functions) {
let str = skillFunc.toString();
if (str.includes('/**')){
docstring += module_name+'.'+skillFunc.name;
docstring += str.substring(str.indexOf('/**')+3, str.indexOf('**/')) + '\n';
if (str.includes('/**')) {
let docEntry = `${module_name}.${skillFunc.name}\n`;
docEntry += str.substring(str.indexOf('/**') + 3, str.indexOf('**/')).trim();
docArray.push(docEntry);
}
}
return docstring;
return docArray;
}
export function getSkillDocs() {
let docstring = "\n*SKILL DOCS\nThese skills are javascript functions that can be called when writing actions and skills.\n";
docstring += docHelper(Object.values(skills), 'skills');
docstring += docHelper(Object.values(world), 'world');
return docstring + '*\n';
let docArray = [];
docArray = docArray.concat(docHelper(Object.values(skills), 'skills'));
docArray = docArray.concat(docHelper(Object.values(world), 'world'));
return docArray;
}

View file

@ -0,0 +1,69 @@
import { cosineSimilarity } from '../../utils/math.js';
import { getSkillDocs } from './index.js';
import { wordOverlapScore } from '../../utils/text.js';
export class SkillLibrary {
constructor(agent,embedding_model) {
this.agent = agent;
this.embedding_model = embedding_model;
this.skill_docs_embeddings = {};
this.skill_docs = null;
}
async initSkillLibrary() {
const skillDocs = getSkillDocs();
this.skill_docs = skillDocs;
if (this.embedding_model) {
try {
const embeddingPromises = skillDocs.map((doc) => {
return (async () => {
let func_name_desc = doc.split('\n').slice(0, 2).join('');
this.skill_docs_embeddings[doc] = await this.embedding_model.embed(func_name_desc);
})();
});
await Promise.all(embeddingPromises);
} catch (error) {
console.warn('Error with embedding model, using word-overlap instead.');
this.embedding_model = null;
}
}
}
async getAllSkillDocs() {
return this.skill_docs;
}
async getRelevantSkillDocs(message, select_num) {
if(!message) // use filler message if none is provided
message = '(no message)';
let skill_doc_similarities = [];
if (!this.embedding_model) {
skill_doc_similarities = Object.keys(this.skill_docs)
.map(doc_key => ({
doc_key,
similarity_score: wordOverlapScore(message, this.skill_docs[doc_key])
}))
.sort((a, b) => b.similarity_score - a.similarity_score);
}
else {
let latest_message_embedding = '';
skill_doc_similarities = Object.keys(this.skill_docs_embeddings)
.map(doc_key => ({
doc_key,
similarity_score: cosineSimilarity(latest_message_embedding, this.skill_docs_embeddings[doc_key])
}))
.sort((a, b) => b.similarity_score - a.similarity_score);
}
let length = skill_doc_similarities.length;
if (typeof select_num !== 'number' || isNaN(select_num) || select_num < 0) {
select_num = length;
} else {
select_num = Math.min(Math.floor(select_num), length);
}
let selected_docs = skill_doc_similarities.slice(0, select_num);
let relevant_skill_docs = '#### RELEVENT DOCS INFO ###\nThe following functions are listed in descending order of relevance.\n';
relevant_skill_docs += 'SkillDocs:\n'
relevant_skill_docs += selected_docs.map(doc => `${doc.doc_key}`).join('\n### ');
return relevant_skill_docs;
}
}

View file

@ -79,7 +79,7 @@ export async function craftRecipe(bot, itemName, num=1) {
}
}
if (!recipes || recipes.length === 0) {
log(bot, `You do not have the resources to craft a ${itemName}. It requires: ${Object.entries(mc.getItemCraftingRecipes(itemName)[0]).map(([key, value]) => `${key}: ${value}`).join(', ')}.`);
log(bot, `You do not have the resources to craft a ${itemName}. It requires: ${Object.entries(mc.getItemCraftingRecipes(itemName)[0][0]).map(([key, value]) => `${key}: ${value}`).join(', ')}.`);
if (placedTable) {
await collectBlock(bot, 'crafting_table', 1);
}
@ -111,6 +111,18 @@ export async function craftRecipe(bot, itemName, num=1) {
return true;
}
export async function wait(seconds) {
/**
* Waits for the given number of seconds.
* @param {number} seconds, the number of seconds to wait.
* @returns {Promise<boolean>} true if the wait was successful, false otherwise.
* @example
* await skills.wait(10);
**/
// setTimeout is disabled to prevent unawaited code, so this is a safe alternative
await new Promise(resolve => setTimeout(resolve, seconds * 1000));
return true;
}
export async function smeltItem(bot, itemName, num=1) {
/**
@ -558,6 +570,14 @@ export async function placeBlock(bot, blockType, x, y, z, placeOn='bottom', dont
const target_dest = new Vec3(Math.floor(x), Math.floor(y), Math.floor(z));
if (bot.modes.isOn('cheat') && !dontCheat) {
if (bot.restrict_to_inventory) {
let block = bot.inventory.items().find(item => item.name === blockType);
if (!block) {
log(bot, `Cannot place ${blockType}, you are restricted to your current inventory.`);
return false;
}
}
// invert the facing direction
let face = placeOn === 'north' ? 'south' : placeOn === 'south' ? 'north' : placeOn === 'east' ? 'west' : 'east';
if (blockType.includes('torch') && placeOn !== 'bottom') {
@ -599,7 +619,7 @@ export async function placeBlock(bot, blockType, x, y, z, placeOn='bottom', dont
if (item_name == "redstone_wire")
item_name = "redstone";
let block = bot.inventory.items().find(item => item.name === item_name);
if (!block && bot.game.gameMode === 'creative') {
if (!block && bot.game.gameMode === 'creative' && !bot.restrict_to_inventory) {
await bot.creative.setInventorySlot(36, mc.makeItem(item_name, 1)); // 36 is first hotbar slot
block = bot.inventory.items().find(item => item.name === item_name);
}
@ -897,18 +917,32 @@ export async function giveToPlayer(bot, itemType, username, num=1) {
}
// if we are too close, make some distance
if (bot.entity.position.distanceTo(player.position) < 2) {
let goal = new pf.goals.GoalNear(player.position.x, player.position.y, player.position.z, 2);
let inverted_goal = new pf.goals.GoalInvert(goal);
bot.pathfinder.setMovements(new pf.Movements(bot));
await bot.pathfinder.goto(inverted_goal);
let too_close = true;
let start_moving_away = Date.now();
await moveAwayFromEntity(bot, player, 2);
while (too_close && !bot.interrupt_code) {
await new Promise(resolve => setTimeout(resolve, 500));
too_close = bot.entity.position.distanceTo(player.position) < 5;
if (too_close) {
await moveAwayFromEntity(bot, player, 5);
}
if (Date.now() - start_moving_away > 3000) {
break;
}
}
if (too_close) {
log(bot, `Failed to give ${itemType} to ${username}, too close.`);
return false;
}
}
await bot.lookAt(player.position);
if (await discard(bot, itemType, num)) {
let given = false;
bot.once('playerCollect', (collector, collected) => {
console.log(collected.name);
if (collector.username === username) {
log(bot, `${username} recieved ${itemType}.`);
log(bot, `${username} received ${itemType}.`);
given = true;
}
});
@ -1106,6 +1140,21 @@ export async function moveAway(bot, distance) {
return true;
}
export async function moveAwayFromEntity(bot, entity, distance=16) {
/**
* Move away from the given entity.
* @param {MinecraftBot} bot, reference to the minecraft bot.
* @param {Entity} entity, the entity to move away from.
* @param {number} distance, the distance to move away.
* @returns {Promise<boolean>} true if the bot moved away, false otherwise.
**/
let goal = new pf.goals.GoalFollow(entity, distance);
let inverted_goal = new pf.goals.GoalInvert(goal);
bot.pathfinder.setMovements(new pf.Movements(bot));
await bot.pathfinder.goto(inverted_goal);
return true;
}
export async function avoidEnemies(bot, distance=16) {
/**
* Move a given distance away from all nearby enemy mobs.
@ -1247,14 +1296,25 @@ export async function tillAndSow(bot, x, y, z, seedType=null) {
* @returns {Promise<boolean>} true if the ground was tilled, false otherwise.
* @example
* let position = world.getPosition(bot);
* await skills.till(bot, position.x, position.y - 1, position.x);
* await skills.tillAndSow(bot, position.x, position.y - 1, position.x, "wheat");
**/
console.log(x, y, z)
x = Math.round(x);
y = Math.round(y);
z = Math.round(z);
let block = bot.blockAt(new Vec3(x, y, z));
console.log(x, y, z)
if (bot.modes.isOn('cheat')) {
let to_remove = ['_seed', '_seeds'];
for (let remove of to_remove) {
if (seedType.endsWith(remove)) {
seedType = seedType.replace(remove, '');
}
}
placeBlock(bot, 'farmland', x, y, z);
placeBlock(bot, seedType, x, y+1, z);
return true;
}
if (block.name !== 'grass_block' && block.name !== 'dirt' && block.name !== 'farmland') {
log(bot, `Cannot till ${block.name}, must be grass_block or dirt.`);
return false;

View file

@ -39,6 +39,82 @@ export function getNearestFreeSpace(bot, size=1, distance=8) {
}
export function getBlockAtPosition(bot, x=0, y=0, z=0) {
/**
* Get a block from the bot's relative position
* @param {Bot} bot - The bot to get the block for.
* @param {number} x - The relative x offset to serach, default 0.
* @param {number} y - The relative y offset to serach, default 0.
* @param {number} y - The relative z offset to serach, default 0.
* @returns {Block} - The nearest block.
* @example
* let blockBelow = world.getBlockAtPosition(bot, 0, -1, 0);
* let blockAbove = world.getBlockAtPosition(bot, 0, 2, 0); since minecraft position is at the feet
**/
let block = bot.blockAt(bot.entity.position.offset(x, y, z));
if (!block) block = {name: 'air'};
return block;
}
export function getSurroundingBlocks(bot) {
/**
* Get the surrounding blocks from the bot's environment.
* @param {Bot} bot - The bot to get the block for.
* @returns {string[]} - A list of block results as strings.
* @example
**/
// Create a list of block position results that can be unpacked.
let res = [];
res.push(`Block Below: ${getBlockAtPosition(bot, 0, -1, 0).name}`);
res.push(`Block at Legs: ${getBlockAtPosition(bot, 0, 0, 0).name}`);
res.push(`Block at Head: ${getBlockAtPosition(bot, 0, 1, 0).name}`);
return res;
}
export function getFirstBlockAboveHead(bot, ignore_types=null, distance=32) {
/**
* Searches a column from the bot's position for the first solid block above its head
* @param {Bot} bot - The bot to get the block for.
* @param {string[]} ignore_types - The names of the blocks to ignore.
* @param {number} distance - The maximum distance to search, default 32.
* @returns {string} - The fist block above head.
* @example
* let firstBlockAboveHead = world.getFirstBlockAboveHead(bot, null, 32);
**/
// if ignore_types is not a list, make it a list.
let ignore_blocks = [];
if (ignore_types === null) ignore_blocks = ['air', 'cave_air'];
else {
if (!Array.isArray(ignore_types))
ignore_types = [ignore_types];
for(let ignore_type of ignore_types) {
if (mc.getBlockId(ignore_type)) ignore_blocks.push(ignore_type);
}
}
// The block above, stops when it finds a solid block .
let block_above = {name: 'air'};
let height = 0
for (let i = 0; i < distance; i++) {
let block = bot.blockAt(bot.entity.position.offset(0, i+2, 0));
if (!block) block = {name: 'air'};
// Ignore and continue
if (ignore_blocks.includes(block.name)) continue;
// Defaults to any block
block_above = block;
height = i;
break;
}
if (ignore_blocks.includes(block_above.name)) return 'none';
return `${block_above.name} (${height} blocks up)`;
}
export function getNearestBlocks(bot, block_types=null, distance=16, count=10000) {
/**
* Get a list of the nearest blocks of the given types.

View file

@ -277,7 +277,7 @@ const modes_list = [
];
async function execute(mode, agent, func, timeout=-1) {
if (agent.self_prompter.on)
if (agent.self_prompter.isActive())
agent.self_prompter.stopLoop();
let interrupted_action = agent.actions.currentActionLabel;
mode.active = true;
@ -290,7 +290,7 @@ async function execute(mode, agent, func, timeout=-1) {
let should_reprompt =
interrupted_action && // it interrupted a previous action
!agent.actions.resume_func && // there is no resume function
!agent.self_prompter.on && // self prompting is not on
!agent.self_prompter.isActive() && // self prompting is not on
!code_return.interrupted; // this mode action was not interrupted by something else
if (should_reprompt) {
@ -311,9 +311,9 @@ for (let mode of modes_list) {
class ModeController {
/*
SECURITY WARNING:
ModesController must be isolated. Do not store references to external objects like `agent`.
ModesController must be reference isolated. Do not store references to external objects like `agent`.
This object is accessible by LLM generated code, so any stored references are also accessible.
This can be used to expose sensitive information by malicious human prompters.
This can be used to expose sensitive information by malicious prompters.
*/
constructor() {
this.behavior_log = '';
@ -404,6 +404,9 @@ export function initModes(agent) {
_agent = agent;
// the mode controller is added to the bot object so it is accessible from anywhere the bot is used
agent.bot.modes = new ModeController();
if (agent.task) {
agent.bot.restrict_to_inventory = agent.task.restrict_to_inventory;
}
let modes_json = agent.prompter.getInitModes();
if (modes_json) {
agent.bot.modes.loadJson(modes_json);

View file

@ -204,7 +204,7 @@ class ItemWrapper {
}
createChildren() {
let recipes = mc.getItemCraftingRecipes(this.name);
let recipes = mc.getItemCraftingRecipes(this.name).map(([recipe, craftedCount]) => recipe);
if (recipes) {
for (let recipe of recipes) {
let includes_blacklisted = false;

View file

@ -1,7 +1,10 @@
const STOPPED = 0
const ACTIVE = 1
const PAUSED = 2
export class SelfPrompter {
constructor(agent) {
this.agent = agent;
this.on = false;
this.state = STOPPED;
this.loop_active = false;
this.interrupt = false;
this.prompt = '';
@ -16,16 +19,38 @@ export class SelfPrompter {
return 'No prompt specified. Ignoring request.';
prompt = this.prompt;
}
if (this.on) {
this.prompt = prompt;
}
this.on = true;
this.state = ACTIVE;
this.prompt = prompt;
this.startLoop();
}
setPrompt(prompt) {
isActive() {
return this.state === ACTIVE;
}
isStopped() {
return this.state === STOPPED;
}
isPaused() {
return this.state === PAUSED;
}
async handleLoad(prompt, state) {
if (state == undefined)
state = STOPPED;
this.state = state;
this.prompt = prompt;
if (state !== STOPPED && !prompt)
throw new Error('No prompt loaded when self-prompting is active');
if (state === ACTIVE) {
await this.start(prompt);
}
}
setPromptPaused(prompt) {
this.prompt = prompt;
this.state = PAUSED;
}
async startLoop() {
@ -38,7 +63,7 @@ export class SelfPrompter {
let no_command_count = 0;
const MAX_NO_COMMAND = 3;
while (!this.interrupt) {
const msg = `You are self-prompting with the goal: '${this.prompt}'. Your next response MUST contain a command !withThisSyntax. Respond:`;
const msg = `You are self-prompting with the goal: '${this.prompt}'. Your next response MUST contain a command with this syntax: !commandName. Respond:`;
let used_command = await this.agent.handleMessage('system', msg, -1);
if (!used_command) {
@ -47,7 +72,7 @@ export class SelfPrompter {
let out = `Agent did not use command in the last ${MAX_NO_COMMAND} auto-prompts. Stopping auto-prompting.`;
this.agent.openChat(out);
console.warn(out);
this.on = false;
this.state = STOPPED;
break;
}
}
@ -63,7 +88,7 @@ export class SelfPrompter {
update(delta) {
// automatically restarts loop
if (this.on && !this.loop_active && !this.interrupt) {
if (this.state === ACTIVE && !this.loop_active && !this.interrupt) {
if (this.agent.isIdle())
this.idle_time += delta;
else
@ -96,12 +121,19 @@ export class SelfPrompter {
this.interrupt = true;
if (stop_action)
await this.agent.actions.stop();
await this.stopLoop();
this.on = false;
this.stopLoop();
this.state = STOPPED;
}
async pause() {
this.interrupt = true;
await this.agent.actions.stop();
this.stopLoop();
this.state = PAUSED;
}
shouldInterrupt(is_self_prompt) { // to be called from handleMessage
return is_self_prompt && this.on && this.interrupt;
return is_self_prompt && (this.state === ACTIVE || this.state === PAUSED) && this.interrupt;
}
handleUserPromptedCmd(is_self_prompt, is_action) {

View file

@ -62,13 +62,20 @@ export class Task {
}
this.taskTimeout = this.data.timeout || 300;
this.taskStartTime = Date.now();
if (this.task_type === 'construction') {
this.validator = new ConstructionTaskValidator(this.data, this.agent);
} else if (this.task_type === 'techtree') {
this.validator = new CraftTaskValidator(this.data, this.agent);
}
this.blocked_actions = this.data.blocked_actions || [];
if (this.goal)
if (this.data.blocked_actions) {
this.blocked_actions = this.data.blocked_actions[this.agent.count_id.toString()] || [];
} else {
this.blocked_actions = [];
}
this.restrict_to_inventory = !!this.data.restrict_to_inventory;
if (this.data.goal)
this.blocked_actions.push('!endGoal');
if (this.conversation)
this.blocked_actions.push('!endConversation');
@ -99,11 +106,6 @@ export class Task {
isDone() {
if (this.validator && this.validator.validate())
return {"message": 'Task successful', "code": 2};
// TODO check for other terminal conditions
// if (this.task.goal && !this.self_prompter.on)
// return {"message": 'Agent ended goal', "code": 3};
// if (this.task.conversation && !inConversation())
// return {"message": 'Agent ended conversation', "code": 3};
if (this.taskTimeout) {
const elapsedTime = (Date.now() - this.taskStartTime) / 1000;
if (elapsedTime >= this.taskTimeout) {
@ -121,19 +123,23 @@ export class Task {
return;
let bot = this.agent.bot;
let name = this.agent.name;
bot.chat(`/clear ${name}`);
console.log(`Cleared ${name}'s inventory.`);
//kill all drops
if (this.agent.count_id === 0) {
bot.chat(`/kill @e[type=item]`);
}
//wait for a bit so inventory is cleared
await new Promise((resolve) => setTimeout(resolve, 500));
let initial_inventory = null;
if (this.data.agent_count > 1) {
var initial_inventory = this.data.initial_inventory[this.agent.count_id.toString()];
initial_inventory = this.data.initial_inventory[this.agent.count_id.toString()];
console.log("Initial inventory:", initial_inventory);
} else if (this.data) {
console.log("Initial inventory:", this.data.initial_inventory);
var initial_inventory = this.data.initial_inventory;
initial_inventory = this.data.initial_inventory;
}
if ("initial_inventory" in this.data) {
@ -201,7 +207,7 @@ export class Task {
await new Promise((resolve) => setTimeout(resolve, 10000));
if (available_agents.length < this.data.agent_count) {
console.log(`Missing ${this.data.agent_count - available_agents.length} bot(s).`);
this.agent.cleanKill('Not all required players/bots are present in the world. Exiting.', 4);
this.agent.killAll();
}
}

View file

@ -3,8 +3,9 @@ import { strictFormat } from '../utils/text.js';
import { getKey } from '../utils/keys.js';
export class Claude {
constructor(model_name, url) {
constructor(model_name, url, params) {
this.model_name = model_name;
this.params = params || {};
let config = {};
if (url)
@ -20,13 +21,16 @@ export class Claude {
let res = null;
try {
console.log('Awaiting anthropic api response...')
// console.log('Messages:', messages);
if (!this.params.max_tokens) {
this.params.max_tokens = 4096;
}
const resp = await this.anthropic.messages.create({
model: this.model_name || "claude-3-sonnet-20240229",
system: systemMessage,
max_tokens: 2048,
messages: messages,
...(this.params || {})
});
console.log('Received.')
res = resp.content[0].text;
}

58
src/models/deepseek.js Normal file
View file

@ -0,0 +1,58 @@
import OpenAIApi from 'openai';
import { getKey, hasKey } from '../utils/keys.js';
import { strictFormat } from '../utils/text.js';
export class DeepSeek {
constructor(model_name, url, params) {
this.model_name = model_name;
this.params = params;
let config = {};
config.baseURL = url || 'https://api.deepseek.com';
config.apiKey = getKey('DEEPSEEK_API_KEY');
this.openai = new OpenAIApi(config);
}
async sendRequest(turns, systemMessage, stop_seq='***') {
let messages = [{'role': 'system', 'content': systemMessage}].concat(turns);
messages = strictFormat(messages);
const pack = {
model: this.model_name || "deepseek-chat",
messages,
stop: stop_seq,
...(this.params || {})
};
let res = null;
try {
console.log('Awaiting deepseek api response...')
// console.log('Messages:', messages);
let completion = await this.openai.chat.completions.create(pack);
if (completion.choices[0].finish_reason == 'length')
throw new Error('Context length exceeded');
console.log('Received.')
res = completion.choices[0].message.content;
}
catch (err) {
if ((err.message == 'Context length exceeded' || err.code == 'context_length_exceeded') && turns.length > 1) {
console.log('Context length exceeded, trying again with shorter context.');
return await this.sendRequest(turns.slice(1), systemMessage, stop_seq);
} else {
console.log(err);
res = 'My brain disconnected, try again.';
}
}
return res;
}
async embed(text) {
throw new Error('Embeddings are not supported by Deepseek.');
}
}

View file

@ -1,10 +1,11 @@
import { GoogleGenerativeAI } from '@google/generative-ai';
import { toSinglePrompt } from '../utils/text.js';
import { toSinglePrompt, strictFormat } from '../utils/text.js';
import { getKey } from '../utils/keys.js';
export class Gemini {
constructor(model_name, url) {
constructor(model_name, url, params) {
this.model_name = model_name;
this.params = params;
this.url = url;
this.safetySettings = [
{
@ -34,29 +35,47 @@ export class Gemini {
async sendRequest(turns, systemMessage) {
let model;
const modelConfig = {
model: this.model_name || "gemini-1.5-flash",
// systemInstruction does not work bc google is trash
};
if (this.url) {
model = this.genAI.getGenerativeModel(
{ model: this.model_name || "gemini-1.5-flash" },
modelConfig,
{ baseUrl: this.url },
{ safetySettings: this.safetySettings }
);
} else {
model = this.genAI.getGenerativeModel(
{ model: this.model_name || "gemini-1.5-flash" },
modelConfig,
{ safetySettings: this.safetySettings }
);
}
const stop_seq = '***';
const prompt = toSinglePrompt(turns, systemMessage, stop_seq, 'model');
console.log('Awaiting Google API response...');
const result = await model.generateContent(prompt);
turns.unshift({ role: 'system', content: systemMessage });
turns = strictFormat(turns);
let contents = [];
for (let turn of turns) {
contents.push({
role: turn.role === 'assistant' ? 'model' : 'user',
parts: [{ text: turn.content }]
});
}
const result = await model.generateContent({
contents,
generationConfig: {
...(this.params || {})
}
});
const response = await result.response;
const text = response.text();
console.log('Received.');
if (!text.includes(stop_seq)) return text;
const idx = text.indexOf(stop_seq);
return text.slice(0, idx);
return text;
}
async embed(text) {

View file

@ -3,8 +3,9 @@ import { getKey, hasKey } from '../utils/keys.js';
import { strictFormat } from '../utils/text.js';
export class GPT {
constructor(model_name, url) {
constructor(model_name, url, params) {
this.model_name = model_name;
this.params = params;
let config = {};
if (url)
@ -25,6 +26,7 @@ export class GPT {
model: this.model_name || "gpt-3.5-turbo",
messages,
stop: stop_seq,
...(this.params || {})
};
if (this.model_name.includes('o1')) {
pack.messages = strictFormat(messages);
@ -32,8 +34,9 @@ export class GPT {
}
let res = null;
try {
console.log('Awaiting openai api response...')
console.log('Awaiting openai api response from model', this.model_name)
// console.log('Messages:', messages);
let completion = await this.openai.chat.completions.create(pack);
if (completion.choices[0].finish_reason == 'length')
@ -54,6 +57,8 @@ export class GPT {
}
async embed(text) {
if (text.length > 8191)
text = text.slice(0, 8191);
const embedding = await this.openai.embeddings.create({
model: this.model_name || "text-embedding-3-small",
input: text,

View file

@ -3,8 +3,10 @@ import { getKey } from '../utils/keys.js';
// xAI doesn't supply a SDK for their models, but fully supports OpenAI and Anthropic SDKs
export class Grok {
constructor(model_name, url) {
constructor(model_name, url, params) {
this.model_name = model_name;
this.url = url;
this.params = params;
let config = {};
if (url)
@ -23,7 +25,8 @@ export class Grok {
const pack = {
model: this.model_name || "grok-beta",
messages,
stop: [stop_seq]
stop: [stop_seq],
...(this.params || {})
};
let res = null;

View file

@ -4,12 +4,13 @@ import { getKey } from '../utils/keys.js';
// Umbrella class for Mixtral, LLama, Gemma...
export class GroqCloudAPI {
constructor(model_name, url, max_tokens=16384) {
constructor(model_name, url, params) {
this.model_name = model_name;
this.url = url;
this.max_tokens = max_tokens;
this.params = params || {};
// ReplicateAPI theft :3
if (this.url) {
console.warn("Groq Cloud has no implementation for custom URLs. Ignoring provided URL.");
}
this.groq = new Groq({ apiKey: getKey('GROQCLOUD_API_KEY') });
@ -20,14 +21,15 @@ export class GroqCloudAPI {
let res = null;
try {
console.log("Awaiting Groq response...");
if (!this.params.max_tokens) {
this.params.max_tokens = 16384;
}
let completion = await this.groq.chat.completions.create({
"messages": messages,
"model": this.model_name || "mixtral-8x7b-32768",
"temperature": 0.2,
"max_tokens": this.max_tokens, // maximum token limit, differs from model to model
"top_p": 1,
"stream": true,
"stop": stop_seq // "***"
"stop": stop_seq,
...(this.params || {})
});
let temp_res = "";
@ -46,6 +48,6 @@ export class GroqCloudAPI {
}
async embed(text) {
console.log("There is no support for embeddings in Groq support. However, the following text was provided: " + text);
throw new Error('Embeddings are not supported by Groq.');
}
}

View file

@ -3,9 +3,10 @@ import {getKey} from '../utils/keys.js';
import {HfInference} from "@huggingface/inference";
export class HuggingFace {
constructor(model_name, url) {
constructor(model_name, url, params) {
this.model_name = model_name.replace('huggingface/','');
this.url = url;
this.params = params;
if (this.url) {
console.warn("Hugging Face doesn't support custom urls!");
@ -25,7 +26,8 @@ export class HuggingFace {
console.log('Awaiting Hugging Face API response...');
for await (const chunk of this.huggingface.chatCompletionStream({
model: model_name,
messages: [{ role: "user", content: input }]
messages: [{ role: "user", content: input }],
...(this.params || {})
})) {
res += (chunk.choices[0]?.delta?.content || "");
}

View file

@ -1,8 +1,9 @@
import { strictFormat } from '../utils/text.js';
export class Local {
constructor(model_name, url) {
constructor(model_name, url, params) {
this.model_name = model_name;
this.params = params;
this.url = url || 'http://127.0.0.1:11434';
this.chat_endpoint = '/api/chat';
this.embedding_endpoint = '/api/embeddings';
@ -15,7 +16,12 @@ export class Local {
let res = null;
try {
console.log(`Awaiting local response... (model: ${model})`)
res = await this.send(this.chat_endpoint, {model: model, messages: messages, stream: false});
res = await this.send(this.chat_endpoint, {
model: model,
messages: messages,
stream: false,
...(this.params || {})
});
if (res)
res = res['message']['content'];
}

73
src/models/mistral.js Normal file
View file

@ -0,0 +1,73 @@
import { Mistral as MistralClient } from '@mistralai/mistralai';
import { getKey } from '../utils/keys.js';
import { strictFormat } from '../utils/text.js';
export class Mistral {
#client;
constructor(model_name, url, params) {
this.model_name = model_name;
this.params = params;
if (typeof url === "string") {
console.warn("Mistral does not support custom URL's, ignoring!");
}
if (!getKey("MISTRAL_API_KEY")) {
throw new Error("Mistral API Key missing, make sure to set MISTRAL_API_KEY in settings.json")
}
this.#client = new MistralClient(
{
apiKey: getKey("MISTRAL_API_KEY")
}
);
// Prevents the following code from running when model not specified
if (typeof this.model_name === "undefined") return;
// get the model name without the "mistral" or "mistralai" prefix
// e.g "mistral/mistral-large-latest" -> "mistral-large-latest"
if (typeof model_name.split("/")[1] !== "undefined") {
this.model_name = model_name.split("/")[1];
}
}
async sendRequest(turns, systemMessage) {
let result;
try {
const model = this.model_name || "mistral-large-latest";
const messages = [
{ role: "system", content: systemMessage }
];
messages.push(...strictFormat(turns));
const response = await this.#client.chat.complete({
model,
messages,
...(this.params || {})
});
result = response.choices[0].message.content;
} catch (err) {
console.log(err)
result = "My brain disconnected, try again.";
}
return result;
}
async embed(text) {
const embedding = await this.#client.embeddings.create({
model: "mistral-embed",
inputs: text
});
return embedding.data[0].embedding;
}
}

View file

@ -1,11 +1,14 @@
import OpenAIApi from 'openai';
import { getKey } from '../utils/keys.js';
import { strictFormat } from '../utils/text.js';
// llama, mistral
export class Novita {
constructor(model_name, url) {
constructor(model_name, url, params) {
this.model_name = model_name.replace('novita/', '');
this.url = url || 'https://api.novita.ai/v3/openai';
this.params = params;
let config = {
baseURL: this.url
@ -17,10 +20,15 @@ export class Novita {
async sendRequest(turns, systemMessage, stop_seq='***') {
let messages = [{'role': 'system', 'content': systemMessage}].concat(turns);
messages = strictFormat(messages);
const pack = {
model: this.model_name || "meta-llama/llama-3.1-70b-instruct",
messages,
stop: [stop_seq],
...(this.params || {})
};
let res = null;
@ -41,6 +49,18 @@ export class Novita {
res = 'My brain disconnected, try again.';
}
}
if (res.includes('<think>')) {
let start = res.indexOf('<think>');
let end = res.indexOf('</think>') + 8;
if (start != -1) {
if (end != -1) {
res = res.substring(0, start) + res.substring(end);
} else {
res = res.substring(0, start+7);
}
}
res = res.trim();
}
return res;
}

58
src/models/openrouter.js Normal file
View file

@ -0,0 +1,58 @@
import OpenAIApi from 'openai';
import { getKey, hasKey } from '../utils/keys.js';
import { strictFormat } from '../utils/text.js';
export class OpenRouter {
constructor(model_name, url) {
this.model_name = model_name;
let config = {};
config.baseURL = url || 'https://openrouter.ai/api/v1';
const apiKey = getKey('OPENROUTER_API_KEY');
if (!apiKey) {
console.error('Error: OPENROUTER_API_KEY not found. Make sure it is set properly.');
}
// Pass the API key to OpenAI compatible Api
config.apiKey = apiKey;
this.openai = new OpenAIApi(config);
}
async sendRequest(turns, systemMessage, stop_seq='*') {
let messages = [{ role: 'system', content: systemMessage }, ...turns];
messages = strictFormat(messages);
// Choose a valid model from openrouter.ai (for example, "openai/gpt-4o")
const pack = {
model: this.model_name,
messages,
stop: stop_seq
};
let res = null;
try {
console.log('Awaiting openrouter api response...');
let completion = await this.openai.chat.completions.create(pack);
if (!completion?.choices?.[0]) {
console.error('No completion or choices returned:', completion);
return 'No response received.';
}
if (completion.choices[0].finish_reason === 'length') {
throw new Error('Context length exceeded');
}
console.log('Received.');
res = completion.choices[0].message.content;
} catch (err) {
console.error('Error while awaiting response:', err);
// If the error indicates a context-length problem, we can slice the turns array, etc.
res = 'My brain disconnected, try again.';
}
return res;
}
async embed(text) {
throw new Error('Embeddings are not supported by Openrouter.');
}
}

View file

@ -1,37 +1,51 @@
import { readFileSync, mkdirSync, writeFileSync} from 'fs';
import { Examples } from '../utils/examples.js';
import { getCommandDocs } from './commands/index.js';
import { getSkillDocs } from './library/index.js';
import { getCommandDocs } from '../agent/commands/index.js';
import { getSkillDocs } from '../agent/library/index.js';
import { SkillLibrary } from "../agent/library/skill_library.js";
import { stringifyTurns } from '../utils/text.js';
import { getCommand } from './commands/index.js';
import { getCommand } from '../agent/commands/index.js';
import settings from '../../settings.js';
import { Gemini } from '../models/gemini.js';
import { GPT } from '../models/gpt.js';
import { Claude } from '../models/claude.js';
import { ReplicateAPI } from '../models/replicate.js';
import { Local } from '../models/local.js';
import { Novita } from '../models/novita.js';
import { GroqCloudAPI } from '../models/groq.js';
import { HuggingFace } from '../models/huggingface.js';
import { Qwen } from "../models/qwen.js";
import { Grok } from "../models/grok.js";
import { Gemini } from './gemini.js';
import { GPT } from './gpt.js';
import { Claude } from './claude.js';
import { Mistral } from './mistral.js';
import { ReplicateAPI } from './replicate.js';
import { Local } from './local.js';
import { Novita } from './novita.js';
import { GroqCloudAPI } from './groq.js';
import { HuggingFace } from './huggingface.js';
import { Qwen } from "./qwen.js";
import { Grok } from "./grok.js";
import { DeepSeek } from './deepseek.js';
import { OpenRouter } from './openrouter.js';
export class Prompter {
constructor(agent, fp) {
this.agent = agent;
this.profile = JSON.parse(readFileSync(fp, 'utf8'));
this.default_profile = JSON.parse(readFileSync('./profiles/_default.json', 'utf8'));
let default_profile = JSON.parse(readFileSync('./profiles/defaults/_default.json', 'utf8'));
let base_fp = settings.base_profile;
let base_profile = JSON.parse(readFileSync(base_fp, 'utf8'));
for (let key in this.default_profile) {
if (this.profile[key] === undefined)
this.profile[key] = this.default_profile[key];
// first use defaults to fill in missing values in the base profile
for (let key in default_profile) {
if (base_profile[key] === undefined)
base_profile[key] = default_profile[key];
}
// then use base profile to fill in missing values in the individual profile
for (let key in base_profile) {
if (this.profile[key] === undefined)
this.profile[key] = base_profile[key];
}
// base overrides default, individual overrides base
this.convo_examples = null;
this.coding_examples = null;
let name = this.profile.name;
let chat = this.profile.model;
this.cooldown = this.profile.cooldown ? this.profile.cooldown : 0;
this.last_prompt_time = 0;
this.awaiting_coding = false;
@ -40,60 +54,22 @@ export class Prompter {
let max_tokens = null;
if (this.profile.max_tokens)
max_tokens = this.profile.max_tokens;
if (typeof chat === 'string' || chat instanceof String) {
chat = {model: chat};
if (chat.model.includes('gemini'))
chat.api = 'google';
else if (chat.model.includes('gpt') || chat.model.includes('o1'))
chat.api = 'openai';
else if (chat.model.includes('claude'))
chat.api = 'anthropic';
else if (chat.model.includes('huggingface/'))
chat.api = "huggingface";
else if (chat.model.includes('meta/') || chat.model.includes('mistralai/') || chat.model.includes('replicate/'))
chat.api = 'replicate';
else if (chat.model.includes("groq/") || chat.model.includes("groqcloud/"))
chat.api = 'groq';
else if (chat.model.includes('novita/'))
chat.api = 'novita';
else if (chat.model.includes('qwen'))
chat.api = 'qwen';
else if (chat.model.includes('grok'))
chat.api = 'xai';
else
chat.api = 'ollama';
}
console.log('Using chat settings:', chat);
let chat_model_profile = this._selectAPI(this.profile.model);
this.chat_model = this._createModel(chat_model_profile);
if (chat.api === 'google')
this.chat_model = new Gemini(chat.model, chat.url);
else if (chat.api === 'openai')
this.chat_model = new GPT(chat.model, chat.url);
else if (chat.api === 'anthropic')
this.chat_model = new Claude(chat.model, chat.url);
else if (chat.api === 'replicate')
this.chat_model = new ReplicateAPI(chat.model, chat.url);
else if (chat.api === 'ollama')
this.chat_model = new Local(chat.model, chat.url);
else if (chat.api === 'groq') {
this.chat_model = new GroqCloudAPI(chat.model.replace('groq/', '').replace('groqcloud/', ''), chat.url, max_tokens ? max_tokens : 8192);
if (this.profile.code_model) {
let code_model_profile = this._selectAPI(this.profile.code_model);
this.code_model = this._createModel(code_model_profile);
}
else {
this.code_model = this.chat_model;
}
else if (chat.api === 'huggingface')
this.chat_model = new HuggingFace(chat.model, chat.url);
else if (chat.api === 'novita')
this.chat_model = new Novita(chat.model.replace('novita/', ''), chat.url);
else if (chat.api === 'qwen')
this.chat_model = new Qwen(chat.model, chat.url);
else if (chat.api === 'xai')
this.chat_model = new Grok(chat.model, chat.url);
else
throw new Error('Unknown API:', api);
let embedding = this.profile.embedding;
if (embedding === undefined) {
if (chat.api !== 'ollama')
embedding = {api: chat.api};
if (chat_model_profile.api !== 'ollama')
embedding = {api: chat_model_profile.api};
else
embedding = {api: 'none'};
}
@ -113,17 +89,24 @@ export class Prompter {
this.embedding_model = new Local(embedding.model, embedding.url);
else if (embedding.api === 'qwen')
this.embedding_model = new Qwen(embedding.model, embedding.url);
else if (embedding.api === 'mistral')
this.embedding_model = new Mistral(embedding.model, embedding.url);
else if (embedding.api === 'huggingface')
this.embedding_model = new HuggingFace(embedding.model, embedding.url);
else if (embedding.api === 'novita')
this.embedding_model = new Novita(embedding.model, embedding.url);
else {
this.embedding_model = null;
console.log('Unknown embedding: ', embedding ? embedding.api : '[NOT SPECIFIED]', '. Using word overlap.');
let embedding_name = embedding ? embedding.api : '[NOT SPECIFIED]'
console.warn('Unsupported embedding: ' + embedding_name + '. Using word-overlap instead, expect reduced performance. Recommend using a supported embedding model. See Readme.');
}
}
catch (err) {
console.log('Warning: Failed to initialize embedding model:', err.message);
console.log('Continuing anyway, using word overlap instead.');
console.warn('Warning: Failed to initialize embedding model:', err.message);
console.log('Continuing anyway, using word-overlap instead.');
this.embedding_model = null;
}
this.skill_libary = new SkillLibrary(agent, this.embedding_model);
mkdirSync(`./bots/${name}`, { recursive: true });
writeFileSync(`./bots/${name}/last_profile.json`, JSON.stringify(this.profile, null, 4), (err) => {
if (err) {
@ -133,6 +116,76 @@ export class Prompter {
});
}
_selectAPI(profile) {
if (typeof profile === 'string' || profile instanceof String) {
profile = {model: profile};
}
if (!profile.api) {
if (profile.model.includes('gemini'))
profile.api = 'google';
else if (profile.model.includes('openrouter/'))
profile.api = 'openrouter'; // must do before others bc shares model names
else if (profile.model.includes('gpt') || profile.model.includes('o1')|| profile.model.includes('o3'))
profile.api = 'openai';
else if (profile.model.includes('claude'))
profile.api = 'anthropic';
else if (profile.model.includes('huggingface/'))
profile.api = "huggingface";
else if (profile.model.includes('replicate/'))
profile.api = 'replicate';
else if (profile.model.includes('mistralai/') || profile.model.includes("mistral/"))
model_profile.api = 'mistral';
else if (profile.model.includes("groq/") || profile.model.includes("groqcloud/"))
profile.api = 'groq';
else if (profile.model.includes('novita/'))
profile.api = 'novita';
else if (profile.model.includes('qwen'))
profile.api = 'qwen';
else if (profile.model.includes('grok'))
profile.api = 'xai';
else if (profile.model.includes('deepseek'))
profile.api = 'deepseek';
else if (profile.model.includes('llama3'))
profile.api = 'ollama';
else
throw new Error('Unknown model:', profile.model);
}
return profile;
}
_createModel(profile) {
let model = null;
if (profile.api === 'google')
model = new Gemini(profile.model, profile.url, profile.params);
else if (profile.api === 'openai')
model = new GPT(profile.model, profile.url, profile.params);
else if (profile.api === 'anthropic')
model = new Claude(profile.model, profile.url, profile.params);
else if (profile.api === 'replicate')
model = new ReplicateAPI(profile.model.replace('replicate/', ''), profile.url, profile.params);
else if (profile.api === 'ollama')
model = new Local(profile.model, profile.url, profile.params);
else if (profile.api === 'mistral')
model = new Mistral(profile.model, profile.url, profile.params);
else if (profile.api === 'groq')
model = new GroqCloudAPI(profile.model.replace('groq/', '').replace('groqcloud/', ''), profile.url, profile.params);
else if (profile.api === 'huggingface')
model = new HuggingFace(profile.model, profile.url, profile.params);
else if (profile.api === 'novita')
model = new Novita(profile.model.replace('novita/', ''), profile.url, profile.params);
else if (profile.api === 'qwen')
model = new Qwen(profile.model, profile.url, profile.params);
else if (profile.api === 'xai')
model = new Grok(profile.model, profile.url, profile.params);
else if (profile.api === 'deepseek')
model = new DeepSeek(profile.model, profile.url, profile.params);
else if (profile.api === 'openrouter')
model = new OpenRouter(profile.model.replace('openrouter/', ''), profile.url, profile.params);
else
throw new Error('Unknown API:', profile.api);
return model;
}
getName() {
return this.profile.name;
}
@ -143,19 +196,26 @@ export class Prompter {
async initExamples() {
try {
this.convo_examples = new Examples(this.embedding_model);
this.coding_examples = new Examples(this.embedding_model);
this.convo_examples = new Examples(this.embedding_model, settings.num_examples);
this.coding_examples = new Examples(this.embedding_model, settings.num_examples);
// Wait for both examples to load before proceeding
await Promise.all([
this.convo_examples.load(this.profile.conversation_examples),
this.coding_examples.load(this.profile.coding_examples)
]);
this.coding_examples.load(this.profile.coding_examples),
this.skill_libary.initSkillLibrary()
]).catch(error => {
// Preserve error details
console.error('Failed to initialize examples. Error details:', error);
console.error('Stack trace:', error.stack);
throw error;
});
console.log('Examples initialized.');
} catch (error) {
console.error('Failed to initialize examples:', error);
throw error;
console.error('Stack trace:', error.stack);
throw error; // Re-throw with preserved details
}
}
@ -174,7 +234,18 @@ export class Prompter {
prompt = prompt.replaceAll('$ACTION', this.agent.actions.currentActionLabel);
}
if (prompt.includes('$COMMAND_DOCS'))
prompt = prompt.replaceAll('$COMMAND_DOCS', getCommandDocs(this.agent.blocked_actions));
prompt = prompt.replaceAll('$COMMAND_DOCS', getCommandDocs());
if (prompt.includes('$CODE_DOCS')) {
const code_task_content = messages.slice().reverse().find(msg =>
msg.role !== 'system' && msg.content.includes('!newAction(')
)?.content?.match(/!newAction\((.*?)\)/)?.[1] || '';
prompt = prompt.replaceAll(
'$CODE_DOCS',
await this.skill_libary.getRelevantSkillDocs(code_task_content, settings.relevant_docs_count)
);
}
prompt = prompt.replaceAll('$COMMAND_DOCS', getCommandDocs());
if (prompt.includes('$CODE_DOCS'))
prompt = prompt.replaceAll('$CODE_DOCS', getSkillDocs());
if (prompt.includes('$EXAMPLES') && examples !== null)
@ -186,7 +257,8 @@ export class Prompter {
if (prompt.includes('$CONVO'))
prompt = prompt.replaceAll('$CONVO', 'Recent conversation:\n' + stringifyTurns(messages));
if (prompt.includes('$SELF_PROMPT')) {
let self_prompt = this.agent.self_prompter.on ? `YOUR CURRENT ASSIGNED GOAL: "${this.agent.self_prompter.prompt}"\n` : '';
// if active or paused, show the current goal
let self_prompt = !this.agent.self_prompter.isStopped() ? `YOUR CURRENT ASSIGNED GOAL: "${this.agent.self_prompter.prompt}"\n` : '';
prompt = prompt.replaceAll('$SELF_PROMPT', self_prompt);
}
if (prompt.includes('$LAST_GOALS')) {
@ -230,6 +302,9 @@ export class Prompter {
let current_msg_time = this.most_recent_msg_time;
for (let i = 0; i < 3; i++) { // try 3 times to avoid hallucinations
await this.checkCooldown();
if (current_msg_time !== this.most_recent_msg_time) {
return '';
}
let prompt = this.profile.conversing;
prompt = await this.replaceStrings(prompt, messages, this.convo_examples);
let generation = await this.chat_model.sendRequest(messages, prompt);
@ -240,7 +315,7 @@ export class Prompter {
continue;
}
if (current_msg_time !== this.most_recent_msg_time) {
console.warn(this.agent.name + ' recieved new message while generating, discarding old response.');
console.warn(this.agent.name + ' received new message while generating, discarding old response.');
return '';
}
return generation;
@ -257,7 +332,7 @@ export class Prompter {
await this.checkCooldown();
let prompt = this.profile.coding;
prompt = await this.replaceStrings(prompt, messages, this.coding_examples);
let resp = await this.chat_model.sendRequest(messages, prompt);
let resp = await this.code_model.sendRequest(messages, prompt);
this.awaiting_coding = false;
return resp;
}

View file

@ -1,104 +1,79 @@
// This code uses Dashscope and HTTP to ensure the latest support for the Qwen model.
// Qwen is also compatible with the OpenAI API format;
import { getKey } from '../utils/keys.js';
import OpenAIApi from 'openai';
import { getKey, hasKey } from '../utils/keys.js';
import { strictFormat } from '../utils/text.js';
export class Qwen {
constructor(modelName, url) {
this.modelName = modelName;
this.url = url || 'https://dashscope.aliyuncs.com/api/v1/services/aigc/text-generation/generation';
this.apiKey = getKey('QWEN_API_KEY');
constructor(model_name, url, params) {
this.model_name = model_name;
this.params = params;
let config = {};
config.baseURL = url || 'https://dashscope.aliyuncs.com/compatible-mode/v1';
config.apiKey = getKey('QWEN_API_KEY');
this.openai = new OpenAIApi(config);
}
async sendRequest(turns, systemMessage, stopSeq = '***', retryCount = 0) {
if (retryCount > 5) {
console.error('Maximum retry attempts reached.');
return 'Error: Too many retry attempts.';
}
async sendRequest(turns, systemMessage, stop_seq='***') {
let messages = [{'role': 'system', 'content': systemMessage}].concat(turns);
const data = {
model: this.modelName || 'qwen-plus',
input: { messages: [{ role: 'system', content: systemMessage }, ...turns] },
parameters: { result_format: 'message', stop: stopSeq },
messages = strictFormat(messages);
const pack = {
model: this.model_name || "qwen-plus",
messages,
stop: stop_seq,
...(this.params || {})
};
// Add default user message if all messages are 'system' role
if (turns.every((msg) => msg.role === 'system')) {
data.input.messages.push({ role: 'user', content: 'hello' });
}
if (!data.model || !data.input || !data.input.messages || !data.parameters) {
console.error('Invalid request data format:', data);
throw new Error('Invalid request data format.');
}
let res = null;
try {
const response = await this._makeHttpRequest(this.url, data);
const choice = response?.output?.choices?.[0];
if (choice?.finish_reason === 'length' && turns.length > 0) {
return this.sendRequest(turns.slice(1), systemMessage, stopSeq, retryCount + 1);
console.log('Awaiting Qwen api response...');
// console.log('Messages:', messages);
let completion = await this.openai.chat.completions.create(pack);
if (completion.choices[0].finish_reason == 'length')
throw new Error('Context length exceeded');
console.log('Received.');
res = completion.choices[0].message.content;
}
catch (err) {
if ((err.message == 'Context length exceeded' || err.code == 'context_length_exceeded') && turns.length > 1) {
console.log('Context length exceeded, trying again with shorter context.');
return await this.sendRequest(turns.slice(1), systemMessage, stop_seq);
} else {
console.log(err);
res = 'My brain disconnected, try again.';
}
return choice?.message?.content || 'No content received.';
} catch (err) {
console.error('Error occurred:', err);
return 'An error occurred, please try again.';
}
return res;
}
// Why random backoff?
// With a 30 requests/second limit on Alibaba Qwen's embedding service,
// random backoff helps maximize bandwidth utilization.
async embed(text) {
if (!text || typeof text !== 'string') {
console.error('Invalid embedding input: text must be a non-empty string.');
return 'Invalid embedding input: text must be a non-empty string.';
}
const data = {
model: 'text-embedding-v2',
input: { texts: [text] },
parameters: { text_type: 'query' },
};
if (!data.model || !data.input || !data.input.texts || !data.parameters) {
console.error('Invalid embedding request data format:', data);
throw new Error('Invalid embedding request data format.');
}
try {
const response = await this._makeHttpRequest(this.url, data);
const embedding = response?.output?.embeddings?.[0]?.embedding;
return embedding || 'No embedding result received.';
} catch (err) {
console.error('Error occurred:', err);
return 'An error occurred, please try again.';
const maxRetries = 5; // Maximum number of retries
for (let retries = 0; retries < maxRetries; retries++) {
try {
const { data } = await this.openai.embeddings.create({
model: this.model_name || "text-embedding-v3",
input: text,
encoding_format: "float",
});
return data[0].embedding;
} catch (err) {
if (err.status === 429) {
// If a rate limit error occurs, calculate the exponential backoff with a random delay (1-5 seconds)
const delay = Math.pow(2, retries) * 1000 + Math.floor(Math.random() * 2000);
// console.log(`Rate limit hit, retrying in ${delay} ms...`);
await new Promise(resolve => setTimeout(resolve, delay)); // Wait for the delay before retrying
} else {
throw err;
}
}
}
// If maximum retries are reached and the request still fails, throw an error
throw new Error('Max retries reached, request failed.');
}
async _makeHttpRequest(url, data) {
const headers = {
'Authorization': `Bearer ${this.apiKey}`,
'Content-Type': 'application/json',
};
const response = await fetch(url, {
method: 'POST',
headers,
body: JSON.stringify(data),
});
if (!response.ok) {
const errorText = await response.text();
console.error(`Request failed, status code ${response.status}: ${response.statusText}`);
console.error('Error response content:', errorText);
throw new Error(`Request failed, status code ${response.status}: ${response.statusText}`);
}
const responseText = await response.text();
try {
return JSON.parse(responseText);
} catch (err) {
console.error('Failed to parse response JSON:', err);
throw new Error('Invalid response JSON format.');
}
}
}
}

View file

@ -4,9 +4,10 @@ import { getKey } from '../utils/keys.js';
// llama, mistral
export class ReplicateAPI {
constructor(model_name, url) {
constructor(model_name, url, params) {
this.model_name = model_name;
this.url = url;
this.params = params;
if (this.url) {
console.warn('Replicate API does not support custom URLs. Ignoring provided URL.');
@ -22,7 +23,11 @@ export class ReplicateAPI {
const prompt = toSinglePrompt(turns, null, stop_seq);
let model_name = this.model_name || 'meta/meta-llama-3-70b-instruct';
const input = { prompt, system_prompt: systemMessage };
const input = {
prompt,
system_prompt: systemMessage,
...(this.params || {})
};
let res = null;
try {
console.log('Awaiting Replicate API response...');

View file

@ -57,11 +57,9 @@ const argv = yargs(args)
const agent = new Agent();
await agent.start(argv.profile, argv.load_memory, argv.init_message, argv.count_id, argv.task_path, argv.task_id);
} catch (error) {
console.error('Failed to start agent process:', {
message: error.message || 'No error message',
stack: error.stack || 'No stack trace',
error: error
});
console.error('Failed to start agent process:');
console.error(error.message);
console.error(error.stack);
process.exit(1);
}
})();

View file

@ -35,6 +35,16 @@ class MainProxy {
this.socket.on('register-agents-success', () => {
console.log('Agents registered');
});
this.socket.on('shutdown', () => {
console.log('Shutting down');
for (let agentName in this.agent_processes) {
this.agent_processes[agentName].stop();
}
setTimeout(() => {
process.exit(0);
}, 2000);
});
}
addAgent(agent) {

View file

@ -101,6 +101,33 @@ export function createMindServer(port = 8080) {
}
});
socket.on('stop-all-agents', () => {
console.log('Killing all agents');
stopAllAgents();
});
socket.on('shutdown', () => {
console.log('Shutting down');
for (let manager of Object.values(agentManagers)) {
manager.emit('shutdown');
}
setTimeout(() => {
process.exit(0);
}, 2000);
});
socket.on('send-message', (agentName, message) => {
if (!inGameAgents[agentName]) {
console.warn(`Agent ${agentName} not logged in, cannot send message via MindServer.`);
return
}
try {
console.log(`Sending message to agent ${agentName}: ${message}`);
inGameAgents[agentName].emit('send-message', agentName, message)
} catch (error) {
console.error('Error: ', error);
}
});
});
server.listen(port, 'localhost', () => {
@ -121,7 +148,16 @@ function agentsUpdate(socket) {
socket.emit('agents-update', agents);
}
function stopAllAgents() {
for (const agentName in inGameAgents) {
let manager = agentManagers[agentName];
if (manager) {
manager.emit('stop-agent', agentName);
}
}
}
// Optional: export these if you need access to them from other files
export const getIO = () => io;
export const getServer = () => server;
export const getConnectedAgents = () => connectedAgents;
export const getConnectedAgents = () => connectedAgents;

View file

@ -80,12 +80,15 @@
${agent.in_game ? `
<button class="stop-btn" onclick="stopAgent('${agent.name}')">Stop</button>
<button class="restart-btn" onclick="restartAgent('${agent.name}')">Restart</button>
<input type="text" id="messageInput" placeholder="Enter a message or command..."></input><button class="start-btn" onclick="sendMessage('${agent.name}', document.getElementById('messageInput').value)">Send</button>
` : `
<button class="start-btn" onclick="startAgent('${agent.name}')">Start</button>
`}
</div>
</div>
`).join('') :
`).join('') +
`<button class="stop-btn" onclick="killAllAgents()">Stop All</button>
<button class="stop-btn" onclick="shutdown()">Shutdown</button>` :
'<div class="agent">No agents connected</div>';
});
@ -100,6 +103,18 @@
function stopAgent(agentName) {
socket.emit('stop-agent', agentName);
}
function killAllAgents() {
socket.emit('stop-all-agents');
}
function shutdown() {
socket.emit('shutdown');
}
function sendMessage(agentName, message) {
socket.emit('send-message', agentName, message)
}
</script>
</body>
</html>
</html>

View file

@ -1,5 +1,5 @@
import { cosineSimilarity } from './math.js';
import { stringifyTurns } from './text.js';
import { stringifyTurns, wordOverlapScore } from './text.js';
export class Examples {
constructor(model, select_num=2) {
@ -18,21 +18,13 @@ export class Examples {
return messages.trim();
}
getWords(text) {
return text.replace(/[^a-zA-Z ]/g, '').toLowerCase().split(' ');
}
wordOverlapScore(text1, text2) {
const words1 = this.getWords(text1);
const words2 = this.getWords(text2);
const intersection = words1.filter(word => words2.includes(word));
return intersection.length / (words1.length + words2.length - intersection.length);
}
async load(examples) {
this.examples = examples;
if (!this.model) return; // Early return if no embedding model
if (this.select_num === 0)
return;
try {
// Create array of promises first
const embeddingPromises = examples.map(example => {
@ -46,12 +38,15 @@ export class Examples {
// Wait for all embeddings to complete
await Promise.all(embeddingPromises);
} catch (err) {
console.warn('Error with embedding model, using word overlap instead:', err);
console.warn('Error with embedding model, using word-overlap instead.');
this.model = null;
}
}
async getRelevant(turns) {
if (this.select_num === 0)
return [];
let turn_text = this.turnsToText(turns);
if (this.model !== null) {
let embedding = await this.model.embed(turn_text);
@ -62,8 +57,8 @@ export class Examples {
}
else {
this.examples.sort((a, b) =>
this.wordOverlapScore(turn_text, this.turnsToText(b)) -
this.wordOverlapScore(turn_text, this.turnsToText(a))
wordOverlapScore(turn_text, this.turnsToText(b)) -
wordOverlapScore(turn_text, this.turnsToText(a))
);
}
let selected = this.examples.slice(0, this.select_num);

View file

@ -18,7 +18,7 @@ const Item = prismarine_items(mc_version);
* @typedef {string} BlockName
*/
export const WOOD_TYPES = ['oak', 'spruce', 'birch', 'jungle', 'acacia', 'dark_oak'];
export const WOOD_TYPES = ['oak', 'spruce', 'birch', 'jungle', 'acacia', 'dark_oak', 'mangrove', 'cherry'];
export const MATCHING_WOOD_BLOCKS = [
'log',
'planks',
@ -190,7 +190,10 @@ export function getItemCraftingRecipes(itemName) {
recipe[ingredientName] = 0;
recipe[ingredientName]++;
}
recipes.push(recipe);
recipes.push([
recipe,
{craftedCount : r.result.count}
]);
}
return recipes;
@ -202,7 +205,7 @@ export function isSmeltable(itemName) {
}
export function getSmeltingFuel(bot) {
let fuel = bot.inventory.items().find(i => i.name === 'coal' || i.name === 'charcoal')
let fuel = bot.inventory.items().find(i => i.name === 'coal' || i.name === 'charcoal' || i.name === 'blaze_rod')
if (fuel)
return fuel;
fuel = bot.inventory.items().find(i => i.name.includes('log') || i.name.includes('planks'))
@ -214,6 +217,8 @@ export function getSmeltingFuel(bot) {
export function getFuelSmeltOutput(fuelName) {
if (fuelName === 'coal' || fuelName === 'charcoal')
return 8;
if (fuelName === 'blaze_rod')
return 12;
if (fuelName.includes('log') || fuelName.includes('planks'))
return 1.5
if (fuelName === 'coal_block')
@ -325,4 +330,156 @@ export function calculateLimitingResource(availableItems, requiredItems, discret
}
if(discrete) num = Math.floor(num);
return {num, limitingResource}
}
let loopingItems = new Set();
export function initializeLoopingItems() {
loopingItems = new Set(['coal',
'wheat',
'diamond',
'emerald',
'raw_iron',
'raw_gold',
'redstone',
'blue_wool',
'packed_mud',
'raw_copper',
'iron_ingot',
'dried_kelp',
'gold_ingot',
'slime_ball',
'black_wool',
'quartz_slab',
'copper_ingot',
'lapis_lazuli',
'honey_bottle',
'rib_armor_trim_smithing_template',
'eye_armor_trim_smithing_template',
'vex_armor_trim_smithing_template',
'dune_armor_trim_smithing_template',
'host_armor_trim_smithing_template',
'tide_armor_trim_smithing_template',
'wild_armor_trim_smithing_template',
'ward_armor_trim_smithing_template',
'coast_armor_trim_smithing_template',
'spire_armor_trim_smithing_template',
'snout_armor_trim_smithing_template',
'shaper_armor_trim_smithing_template',
'netherite_upgrade_smithing_template',
'raiser_armor_trim_smithing_template',
'sentry_armor_trim_smithing_template',
'silence_armor_trim_smithing_template',
'wayfinder_armor_trim_smithing_template']);
}
/**
* Gets a detailed plan for crafting an item considering current inventory
*/
export function getDetailedCraftingPlan(targetItem, count = 1, current_inventory = {}) {
initializeLoopingItems();
if (!targetItem || count <= 0 || !getItemId(targetItem)) {
return "Invalid input. Please provide a valid item name and positive count.";
}
if (isBaseItem(targetItem)) {
const available = current_inventory[targetItem] || 0;
if (available >= count) return "You have all required items already in your inventory!";
return `${targetItem} is a base item, you need to find ${count - available} more in the world`;
}
const inventory = { ...current_inventory };
const leftovers = {};
const plan = craftItem(targetItem, count, inventory, leftovers);
return formatPlan(plan);
}
function isBaseItem(item) {
return loopingItems.has(item) || getItemCraftingRecipes(item) === null;
}
function craftItem(item, count, inventory, leftovers, crafted = { required: {}, steps: [], leftovers: {} }) {
// Check available inventory and leftovers first
const availableInv = inventory[item] || 0;
const availableLeft = leftovers[item] || 0;
const totalAvailable = availableInv + availableLeft;
if (totalAvailable >= count) {
// Use leftovers first, then inventory
const useFromLeft = Math.min(availableLeft, count);
leftovers[item] = availableLeft - useFromLeft;
const remainingNeeded = count - useFromLeft;
if (remainingNeeded > 0) {
inventory[item] = availableInv - remainingNeeded;
}
return crafted;
}
// Use whatever is available
const stillNeeded = count - totalAvailable;
if (availableLeft > 0) leftovers[item] = 0;
if (availableInv > 0) inventory[item] = 0;
if (isBaseItem(item)) {
crafted.required[item] = (crafted.required[item] || 0) + stillNeeded;
return crafted;
}
const recipe = getItemCraftingRecipes(item)?.[0];
if (!recipe) {
crafted.required[item] = stillNeeded;
return crafted;
}
const [ingredients, result] = recipe;
const craftedPerRecipe = result.craftedCount;
const batchCount = Math.ceil(stillNeeded / craftedPerRecipe);
const totalProduced = batchCount * craftedPerRecipe;
// Add excess to leftovers
if (totalProduced > stillNeeded) {
leftovers[item] = (leftovers[item] || 0) + (totalProduced - stillNeeded);
}
// Process each ingredient
for (const [ingredientName, ingredientCount] of Object.entries(ingredients)) {
const totalIngredientNeeded = ingredientCount * batchCount;
craftItem(ingredientName, totalIngredientNeeded, inventory, leftovers, crafted);
}
// Add crafting step
const stepIngredients = Object.entries(ingredients)
.map(([name, amount]) => `${amount * batchCount} ${name}`)
.join(' + ');
crafted.steps.push(`Craft ${stepIngredients} -> ${totalProduced} ${item}`);
return crafted;
}
function formatPlan({ required, steps, leftovers }) {
const lines = [];
if (Object.keys(required).length > 0) {
lines.push('You are missing the following items:');
Object.entries(required).forEach(([item, count]) =>
lines.push(`- ${count} ${item}`));
lines.push('\nOnce you have these items, here\'s your crafting plan:');
} else {
lines.push('You have all items required to craft this item!');
lines.push('Here\'s your crafting plan:');
}
lines.push('');
lines.push(...steps);
if (Object.keys(leftovers).length > 0) {
lines.push('\nYou will have leftover:');
Object.entries(leftovers).forEach(([item, count]) =>
lines.push(`- ${count} ${item}`));
}
return lines.join('\n');
}

View file

@ -26,8 +26,21 @@ export function toSinglePrompt(turns, system=null, stop_seq='***', model_nicknam
return prompt;
}
// ensures stricter turn order for anthropic/llama models
// combines repeated messages from the same role, separates repeat assistant messages with filler user messages
function _getWords(text) {
return text.replace(/[^a-zA-Z ]/g, '').toLowerCase().split(' ');
}
export function wordOverlapScore(text1, text2) {
const words1 = _getWords(text1);
const words2 = _getWords(text2);
const intersection = words1.filter(word => words2.includes(word));
return intersection.length / (words1.length + words2.length - intersection.length);
}
// ensures stricter turn order and roles:
// - system messages are treated as user messages and prefixed with SYSTEM:
// - combines repeated messages from users
// - separates repeat assistant messages with filler user messages
export function strictFormat(turns) {
let prev_role = null;
let messages = [];

File diff suppressed because it is too large Load diff

File diff suppressed because it is too large Load diff

File diff suppressed because it is too large Load diff

View file

@ -26,9 +26,9 @@
</div>
<script>
function updateLayout() {
var width = window.innerWidth;
var height = window.innerHeight;
var iframes = document.querySelectorAll('.iframe-wrapper');
let width = window.innerWidth;
let height = window.innerHeight;
let iframes = document.querySelectorAll('.iframe-wrapper');
if (width > height) {
iframes.forEach(function(iframe) {
iframe.style.width = '50%';
@ -43,10 +43,10 @@
}
window.addEventListener('resize', updateLayout);
window.addEventListener('load', updateLayout);
var iframes = document.querySelectorAll('.iframe-wrapper');
let iframes = document.querySelectorAll('.iframe-wrapper');
iframes.forEach(function(iframe) {
var port = iframe.getAttribute('data-port');
var loaded = false;
let port = iframe.getAttribute('data-port');
let loaded = false;
function checkServer() {
fetch('http://localhost:' + port, { method: 'HEAD' })
.then(function(response) {