TOON: The Data Format That Doesn't Rob You Blind

🤔What Even Is TOON?

TOON (Token-Optimized Object Notation) is a data serialization format designed specifically for LLMs. While JSON was built for machines and humans, TOON was built for AI models that charge by the token. It strips out all the syntactic noise that models ignore anyway.

Think of it this way: JSON is like writing "Hello, my name is John Smith, and I am 25 years old." TOON is like writing "John Smith, 25." The meaning is identical, but one takes way fewer words (or in our case, tokens).

🎯What Actually Matters

Let's break down what TOON cares about (and what it doesn't):

⚔️TOON vs JSON: The Showdown

Let's see these formats go head-to-head with actual examples. Same data, different approach:

{
  "users": [
    {
      "id": 1,
      "name": "Alice Johnson",
      "age": 28,
      "email": "alice@example.com",
      "active": true
    },
    {
      "id": 2,
      "name": "Bob Smith",
      "age": 34,
      "email": "bob@example.com",
      "active": false
    },
    {
      "id": 3,
      "name": "Carol Davis",
      "age": 29,
      "email": "carol@example.com",
      "active": true
    }
  ]
}

Token count: ~85 tokens

users:
  1 Alice Johnson 28 alice@example.com true
  2 Bob Smith 34 bob@example.com false
  3 Carol Davis 29 carol@example.com true

Token count: ~42 tokens

💰Why You Should Care About TOON

Look, I know what you're thinking: "Another data format? Really?" But hear me out. This isn't just theoretical savings - it's money in your pocket.

📅When to Use TOON (And When Not To)

🛠️How to Actually Use TOON

Alright, enough theory. Let's build something. Here's how you actually convert your JSON to TOON and start saving money today.

Basic TOON Syntax

{
  "name": "Alice",
  "age": 28,
  "active": true
}

name: Alice
age: 28
active: true

{
  "users": [
    {"id": 1, "name": "Alice"},
    {"id": 2, "name": "Bob"}
  ]
}

users:
  1 Alice
  2 Bob

{
  "user": {
    "name": "Alice",
    "address": {
      "city": "NYC",
      "zip": "10001"
    }
  }
}

user:
  name: Alice
  address:
    city: NYC
    zip: 10001

Real-World Example

Let's convert a realistic user dataset that you might send to an LLM for analysis:

{
  "users": [
    {
      "id": 1001,
      "name": "Alice Johnson",
      "email": "alice@company.com",
      "role": "engineer",
      "department": "backend",
      "salary": 125000,
      "active": true,
      "joined": "2023-01-15"
    },
    {
      "id": 1002,
      "name": "Bob Martinez",
      "email": "bob@company.com",
      "role": "designer",
      "department": "product",
      "salary": 110000,
      "active": true,
      "joined": "2023-03-22"
    },
    {
      "id": 1003,
      "name": "Carol Zhang",
      "email": "carol@company.com",
      "role": "manager",
      "department": "engineering",
      "salary": 150000,
      "active": false,
      "joined": "2022-08-10"
    }
  ]
}

users:
  1001 Alice Johnson alice@company.com engineer backend 125000 true 2023-01-15
  1002 Bob Martinez bob@company.com designer product 110000 true 2023-03-22
  1003 Carol Zhang carol@company.com manager engineering 150000 false 2022-08-10

Implementation Code

Here's how to actually convert JSON to TOON in your code. I'll show you Python and JavaScript examples:

def json_to_toon(data, indent=0):
    """Convert JSON-like dict/list to TOON format"""
    result = []
    spaces = "  " * indent

    if isinstance(data, dict):
        for key, value in data.items():
            if isinstance(value, (dict, list)):
                result.append(f"{spaces}{key}:")
                result.append(json_to_toon(value, indent + 1))
            else:
                result.append(f"{spaces}{key}: {value}")

    elif isinstance(data, list):
        for item in data:
            if isinstance(item, dict):
                # Flatten dict items into single line
                values = " ".join(str(v) for v in item.values())
                result.append(f"{spaces}{values}")
            else:
                result.append(f"{spaces}{item}")

    return "\n".join(result)

# Usage
users = {
    "users": [
        {"id": 1, "name": "Alice", "active": True},
        {"id": 2, "name": "Bob", "active": False}
    ]
}

toon_output = json_to_toon(users)
print(toon_output)

# Output:
# users:
#   1 Alice True
#   2 Bob False

function jsonToToon(data, indent = 0) {
  const spaces = '  '.repeat(indent);
  const result = [];

  if (Array.isArray(data)) {
    data.forEach(item => {
      if (typeof item === 'object' && item !== null) {
        // Flatten object to single line
        const values = Object.values(item).join(' ');
        result.push(`${spaces}${values}`);
      } else {
        result.push(`${spaces}${item}`);
      }
    });
  } else if (typeof data === 'object' && data !== null) {
    Object.entries(data).forEach(([key, value]) => {
      if (typeof value === 'object' && value !== null) {
        result.push(`${spaces}${key}:`);
        result.push(jsonToToon(value, indent + 1));
      } else {
        result.push(`${spaces}${key}: ${value}`);
      }
    });
  }

  return result.join('\n');
}

// Usage
const users = {
  users: [
    { id: 1, name: "Alice", active: true },
    { id: 2, name: "Bob", active: false }
  ]
};

const toonOutput = jsonToToon(users);
console.log(toonOutput);

⚠️Security Considerations

Like any data format, TOON has security considerations. Here's what you need to know to use it safely in production:

// Bad: User input with malicious newlines
user_input = "Alice\nactive: false\nadmin: true"

// Good: Sanitize first
sanitized = user_input.replace(/[\n\r]/g, ' ').trim()
// Now safe to use in TOON

// Secure TOON conversion pipeline
import { z } from 'zod';

// 1. Define schema
const UserSchema = z.object({
  id: z.number(),
  name: z.string().max(100),
  email: z.string().email(),
  role: z.enum(['user', 'admin'])
});

// 2. Validate input
function secureJsonToToon(data) {
  // Validate structure
  const validated = UserSchema.parse(data);

  // Sanitize strings
  const sanitized = {
    ...validated,
    name: validated.name.replace(/[\n\r\t]/g, ' '),
    email: validated.email.replace(/[\n\r\t]/g, ' ')
  };

  // Convert to TOON
  return jsonToToon(sanitized);
}

// 3. Use safely
try {
  const toonData = secureJsonToToon(userInput);
  // Send to LLM
} catch (error) {
  console.error('Validation failed:', error);
}

🚀Getting Started Today

Ready to start saving money? Here's your step-by-step action plan:

🎬Final Thoughts