AI & ML interests

None defined yet.

Recent Activity

ErenAta00  updated a collection about 7 hours ago
Maverick XR Agent Dataset
ErenAta00  updated a collection about 7 hours ago
Maverick
ErenAta00  updated a dataset about 20 hours ago
ErenAta00/Maverick-XR-Agent-Dataset
View all activity

ErenAta00 
posted an update about 6 hours ago
view post
Post
21
The Maverick XR Agent Dataset is now public.

It is the data behind Maverick 1.0 and 1.1, our offline tool-calling models for Unity XR scenes: the conversations the models were trained on and the test commands they were measured with. Everything is synthetic. A small simulator of Unity-style houses builds each scene and the reference actions, and the results in every conversation come from that simulator.

What is inside:
- 24,761 training conversations for Maverick 1.1 and 20,091 for Maverick 1.0
- The 33 test cells we sealed before training (3,585 commands) and 29 development sets
- Corrections and undo, gaze and pointing, objects in other rooms, short plans, and requests the tools cannot fulfil
- The tool lists, system prompts and the object catalogue

Two things we want to be clear about. The test cells are no longer sealed, so every evaluation row carries a canary string. Please do not train on them. And the card lists the limits plainly: one simulator, mostly templated wording, English only.

License: CC BY-NC 4.0. You can use it for non-commercial work, including training and evaluating models, and you can adapt it. Credit and citation are required. The card has the credit line and the BibTeX entry.

Dataset: ErenAta00/Maverick-XR-Agent-Dataset
Also on Kaggle, with a starter notebook: https://www.kaggle.com/datasets/erenata/maverick-xr-agent-dataset
Models: ErenAta00/Maverick-4B-Unity-XR-Agent-1.1

Made at the Extended Reality Laboratory (XRLab), Manisa Celal Bayar University:
ExtendedRealityLabMCBU
ErenAta00 
posted an update 8 days ago
view post
Post
3,068
Maverick 1.1 is out.

When we released the first version, the feedback was clear: real people do not give one clean command at a time. They change their minds, they point and say "put that on the shelf", they ask for a lamp that is in another room, and sometimes the one speaking is a remote expert helping someone on site. 1.1 is built around that.

- Say "no, the other one" and it undoes the last action and repeats it on the other object. If the effect can't be undone, like a button that was already pressed, it tells you.
- It sees what people are looking at or pointing at, and who is speaking.
- It finds objects in other rooms, goes there and acts, or asks which one if there are two.
- It follows a short plan in one sentence: "open the cabinet, put the book and the alarm clock in it, then close it".

In the clip, a remote expert corrects the model, then asks it to highlight what the technician is looking at.

On test sets we sealed before training and ran once, the new abilities score 93 to 100%, where 1.0 scores 33 to 87%. On sentences written by people (the ALFRED set) the two versions are level, at 83.2% and 83.8%.

Where it still falls short: in unfamiliar situations it tends to act when it should ask, and a remote speaker can act in the scene with no permission check, so your app has to enforce that.

Same 4B model and licence (Apache-2.0), the same 11 tools plus undo, still offline on a 4 GB laptop GPU. The Unity package for the new scene format (1.1.0) is on GitHub: https://github.com/mcbu-xrlab/Maverick-Unity

Model: ErenAta00/Maverick-4B-Unity-XR-Agent-1.1
Article and full demo: https://huggingface.co/blog/ErenAta00/maverick-1-1-undo-pointing-other-rooms

Built at the Extended Reality Laboratory (XRLab), Manisa Celal Bayar University:
ExtendedRealityLabMCBU
  • 5 replies
·
ErenAta00 
updated a Space 18 days ago
ErenAta00 
posted an update about 1 month ago
view post
Post
4,189
Maverick-4B-Unity-XR-Agent is now on Hugging Face.

It's a 4B model that turns spoken or typed English into actions in Unity scenes. Say "put the red mug on the table" or "turn on the lamp", and it returns the tool call your app executes. If a command could mean two objects, it asks which one. If it can't do something, it says so instead of guessing.

Everything runs on the user's machine through llama.cpp: no API key, no internet connection. The Q4_K_M GGUF is 2.5 GB and needs about 3 GB of GPU memory, so it fits on a 4 GB laptop GPU and usually answers in one to three seconds. It is fine-tuned from Qwen3-4B with QLoRA on about 20,000 English conversations.

Results:
- 83.8% on 499 human-written ALFRED instructions (right action on the right object). The base model, Qwen3-4B, scores 57.1%. The strongest of the five other models we tested, from 1.7B to 120B parameters, was Ministral 3 14B at 67.1%.
- 91.7% on object types it never saw in training.
- 97.3% on 440 commands run through a live Unity scene.

There is also a Unity package that starts the model, describes the scene to it and carries out its tool calls. You install it from the Package Manager with a Git URL.

Model: https://huggingface.co/ErenAta00/Maverick-4B-Unity-XR-Agent-GGUF
Unity package: ErenAta00/Maverick-Unity
Full write-up: https://huggingface.co/blog/ErenAta00/maverick-4b-unity-xr-agent

Built at the Extended Reality Laboratory (XRLab), Manisa Celal Bayar University:
ExtendedRealityLabMCBU

Released under Apache-2.0. Feedback and bug reports are welcome in the Community tab.
ErenAta00 
published a Space about 1 month ago
ErenAta00 
updated a Space about 1 month ago