NYC · Stealth · AI Research Lab

The Equator
Intelligence Lab

An applied research lab advancing deterministic evaluation of LLM reasoning. We design protocols, datasets, and measurement tools that make reasoning performance measurable, reproducible, and auditable.

Read the EQUATOR paper →Explore research
Deterministic evaluationReasoning benchmarksHuman-aligned metrics
EQUATOR Framework

Deterministic evaluation of LLM reasoning

Overview

A deterministic framework to evaluate open-ended reasoning with fixed protocols, binary criteria, and auditable logs.

Method

Blind adjudication using small local LLMs with human tie-breakers; preregistered seeds and prompts; grounded retrieval.

Objective

Provide a robust, reproducible alternative to multiple-choice benchmarks that under-measure reasoning.

Featured publication

EQUATOR Evaluator — A deterministic framework for evaluating open-ended LLM reasoning beyond fluency bias

arXiv →
The Equator Intelligence LabDeterministic EvaluationLLM ReasoningReproducible BenchmarksNYC · StealthEQUATOR EvaluatorHuman-Aligned MetricsarXiv · 2501.00257The Equator Intelligence LabDeterministic EvaluationLLM ReasoningReproducible BenchmarksNYC · StealthEQUATOR EvaluatorHuman-Aligned MetricsarXiv · 2501.00257
01 — About

About EQLabs

EQLabs.ai operates at the intersection of AI reasoning, evaluation, and alignment. We develop reproducible methods to measure how models think—not just how they sound.

Founded from research and contributions by alumni of Columbia University and the Vector Institute. Based in NYC; growing a global network via our LinkedIn research community.

Who we help

  • ·AI labs and R&D teams building reasoning-heavy agents
  • ·Applied ML/platform teams needing reproducible LLM QA
  • ·Regulated domains requiring auditable AI evaluations
  • ·Academic groups running LLM research
  • ·Investors in reasoning, interpretability, and AI safety

Mission

"Bring clarity, balance, and accountability to AI by advancing deterministic evaluation frameworks that align machine reasoning with human standards."

Focus

Reasoning & Evaluation

Modality

Open-ended LLMs

Standard

Deterministic metrics

Ethos

Human-aligned

02 — Research

EQUATOR Evaluator

Our paper, EQUATOR Evaluator, formalizes a deterministic approach to judge open-ended reasoning. We prioritize binary criteria, groundedness, reproducibility, and preregistered protocols.

✓
Deterministic scoring beyond surface fluency
✓
Human-validated reference sets & semantic matching
✓
Cost-efficient local evaluators (small LLMs)
✓
Inter-rater agreement checks (kappa) and error taxonomy

Key artifacts

arXiv preprintOpenEvaluation schemaViewFailure taxonomyView
03 — Watch

Watch our explainer

How EQUATOR redefines AI evaluation through deterministic reasoning.

Watch on YouTube ↗
04 — Benchmark

Reasoning benchmark

We're curating a benchmark of simple, real-world problems LLMs often get wrong—exposing gaps between fluency and understanding.

~1,500

Questions

Growing

Expanding significantly

Blind

True blind studies

Validated

Reproducible

We report inter-rater agreement (Cohen's kappa), maintain versioned datasets, and log all trials for auditability.

Evaluation criteria

✓
Correctness (binary)
✓
Completeness (binary)
✓
Groundedness (binary; no unsupported claims)
✓
Units/Format (binary, when applicable)
✓
Safety/Policy (binary)
Press

Latest announcements

Press Release · October 11, 2025

EQLabs Launches Founders Circle

Join early builders and backers; we're pre-seed and welcoming GPU donations 1–2 generations old.

Read →
05 — Investors & Partners

Backing better AI through better measurement

We're pre-seed and actively raising angel funding, partnering with teams who believe better AI starts with better measurement. If you invest in reasoning, interpretability, or AI safety, let's talk.

Email invest@eqlabs.ai →Founders CircleRequest a demo

Thesis

Deterministic evaluation

Moat

Benchmark + schema

Evidence

arXiv + blind studies

Round

Pre-seed (angels/strategics)

Use of funds

Benchmark growth, infra, pilots

Timeline

Intros this month

Looking for founders

We are not hiring. We're recruiting founders to build with us on deterministic evaluation. If this is you, let's talk.

founders@eqlabs.ai →
06 — Contact

Get in touch

We're pre-seed and actively raising. We are not hiring; we're looking for founders and research collaborators focused on deterministic evaluation. If that's you, we'd love to connect.

One-liner

"We're an AI research lab developing deterministic frameworks to evaluate and improve reasoning in large language models."

Mission

Make AI reasoning transparent, measurable, and aligned with human values.

Links