Go to home page
BACK TO HOME
PRODUCT ENGINEERING CASE // ACTIVE

CRAMAI

Multi-agent AI learning assistant

OPEN PRODUCT

FULL PRODUCT DEMO

CHINESE · 01:05

PRODUCT LOOP

01
Bring materials
PDF · VIDEO · LINK
02
Learn with Tutor
READ · ASK · CREATE
03
Practice & test
QUIZ · MOCK EXAM · COACH
04
Track & improve
REPORT · NEXT PLAN
SHIPPED_OUTCOME

I moved the product from beta-code access to a finishable guest exam, then made the item bank the shared data layer for mock exams and staged coaching: real papers to test, server-side anchors to practice, original stems never shown. Variant generation and the QA kernel are owned by teammates.

MY ROLE
Product Lead / Product Engineer
Dec 2025 - Present
OPEN CASE OVERVIEW

CramAI turns PDFs, videos, and links into a conversational, testable, trackable learning loop. I own product direction, priorities, and feedback review while contributing product design, UI/UX, frontend, the guest path, bank schema, coaching state machine, and GA4 usage signals.

A 65-second Chinese walkthrough of CramAI's entry point and core learning experience. An English edition will be added later.

THREE PRODUCT DECISIONS

A feature list says what shipped. A decision says why it looks like this. The three production screens below are evidence, not decoration.

DEC_0101 / 03

Value before authentication

CONSIDERED

A beta-code wall and a full-page login, so accounts come before experience.

CHOSE

/exam/ silently creates Guest. It reuses the real exam UI with a fixed 12 CFA items. GET withholds correct answers. POST grades deterministically on the server and never calls Grade Agent.

WHY

People leave an email after they have seen grading. Guest access raises cost and abuse risk, so Gateway allowlists only the two guest-exam-demo paths, 32KiB, 12 items max. Account-level capabilities stay gated.

The landing is a public trial entry, not a login wall.
The landing is a public trial entry, not a login wall.
DEC_0202 / 03

Outputs live in the conversation

CONSIDERED

A FeaturePanel wall of isolated buttons for summaries, mind maps, flashcards, and quizzes.

CHOSE

Materials enter a Content state machine. Artifacts hang on the same sessionId so reading, asking, and generating stay continuous.

WHY

A growing toolkit raises cognitive load. Isolated buttons make users lose the material they were reading.

The workspace keeps material context and FeaturePanel outputs in one conversation.
The workspace keeps material context and FeaturePanel outputs in one conversation.
DEC_0303 / 03

Operable bank first, then coaching

CONSIDERED

Keep mock exams on demo JSON, explain real bank items in chat, or reuse the analogy API.

CHOSE

Four tables split papers from questions. Coaching is its own domain and does not write exam-mistake tables. createSession never ships the anchor. Frozen three rungs plus L1–L4.

WHY

Without independent question entities there is no reuse, and no way to practice without leaking original stems. Variant generation and Blind Solver / Judge are owned by teammates.

The exam UI runs on published papers, not demo data.
The exam UI runs on published papers, not demo data.

GUEST /exam/ SEVEN STEPS

Phase one never calls AI to write items, grade, or plan. Answers stay in the browser. Gateway allowlists only GET/POST /agent/guest-exam-demo. The seven-step funnel is fail-open. This is not a conversion-rate claim.

01

Entry

Land on /exam/. The URL is fixed; there is no ?demo=1.

02

Dashboard

NewExamDashboard silently creates Guest. Entry cards, wrong-answer sets, recent exams, and the library stay hidden.

03

Start

Reuse ExamImmersionStage. Load 12 CFA singles from guest_full_loop_12_v2.

04

Submit

POST grade: 32KiB, 12 items max. Deterministic scoring. Failures return to the item with answers kept. Fake scores are forbidden.

05

Result

Reuse the live result page. The radar comes from this grade. GET never returned correct answers.

06

Plan

StudyPlanView renders a 5-block template. No AI job.

07

Register

CTA hits the existing /auth?mode=register. No device fingerprint.

TEAM & OWNERSHIP

TEAM SIZE
Cross-functional team, peaking at 9
Product Lead / Product Engineer

I OWNED

Product direction, priorities, and release sequencing
Reviewing market/user feedback and turning it into product decisions
Product development, UI/UX, frontend, guest path, bank schema, and coaching state machine
Event instrumentation and GA4 / Plausible usage signals — not conversion rate

COLLABORATION BOUNDARY

Internal AI-agent algorithms, Blind Solver / Judge, variant generation, and difficulty judging were owned by other team members
Core multimodal RAG capabilities were owned by other team members
AWS infrastructure and deployment were owned by other team members
OPEN ROLE DEFINITION

As Product Lead / Product Engineer, I own direction, priorities, feedback review, and the main product experience while contributing UI/UX, frontend, the guest path, bank schema, coaching state machine, and GA4 usage signals. Internal algorithms for the 12 agents, Blind Solver / Judge, variant generation, multimodal RAG, and AWS CDK are owned by other team members.

REAL USAGE SIGNALS

UPDATED // 2026-09-05
SIG_01
1,264
ACTIVE USERS IN 90 DAYS
SIG_02
22,514
PRODUCT EVENTS
SIG_03
2M 29S
AVG ENGAGEMENT PER ACTIVE USER
SIG_04
1,670
ZH + EN DASHBOARD VIEWS
LOADING_DATA_STREAM...
OPEN METHODOLOGY & DATA NOTE

Early production data shows this is a product people use, not a concept left at prototype stage: users reached the landing, learning workspace, and exam paths.

Period: Jun 7–Sep 4, 2026. These are aggregate GA4 usage signals, not claims of growth, retention, or conversion lift. Funnel conclusions will be updated after event-contract and internal-traffic cleanup.

CURRENT FOCUS

ITEM BANK AS DATA LAYER

Operable question entities first; mock exams and coaching only share anchors after that

WHY THE BANK CAME FIRST

Quick 12 / Full 180 proved the exam experience, but one demo JSON could not support operations: questions could not be reused across papers, algorithm changes could not be replayed, and coaching had no qualified real items to use as anchors. Questions became independent entities; papers only arrange them.

QA SCALE

CFA Level I at about 16 papers / 2,600+ questions is import and QA scale, not items I authored. Image hashes saw 16 reused items; text identity reported 443 cross-paper reuses and 2 text-identical stems with conflicting answers.

PHASE-1 FOUR TABLES

The question is the reusable entity. A paper is a set of references plus arrangement. Correct answers are stored as text, not option letters, because letters are unstable across papers.

01
RawExamImport
Source files land in S3 with dual hashes, so a later normalizer can be replayed instead of discarding bytes after parse.
02
ExamPaper
Owns session, order, year, syllabus version, and paper kind. A historical 240-item paper is not rewritten by a current 180-item blueprint.
03
ExamPaperItem
Arrangement on the paper: question number, item-set, source passage, and points. Vignette structure stays here so identical stems under different passages do not collapse.
04
ExamQuestion
canonical_hash for identity, revision_hash for version. Subjective grading payloads live in content_payload. Seven canonical types share one question table.

EXAM LIBRARY · Category → Program → Track → generation template → Card/Paper → Question + Scope. Templates are generation specs, not constraints on historical papers. Retrieval is scoped to the selected track and fails closed.

GUARDED IMPORT, READ-ONLY BY DEFAULT

Conversion stops at canonical JSON and never touches the database. Audit and preview stay read-only until an explicit apply, then a separate publish.

01
CONVERT

A converter registry covers schema JSON, Xingce PDFs, CFA paired PDFs, and Shenlun combined PDFs. Artifacts cache on converter version and source SHA-256.

02
AUDIT

Local audit mirrors Admin rejections: 10 MB, required explanations, paperKind enum. Text identity ignores crops so cross-paper reuse and answer conflicts surface.

03
PREVIEW

Remote preview in waves, resumable by checksum. A batch parser failure re-parses each file and names the rejects instead of one opaque error.

04
APPLY / PUBLISH

--apply writes; --publish publishes. Policy: never change or invent answer keys; label AI explanations; prefer original PDFs over earlier JSON.

SEVEN CANONICAL TYPES · Single, multiple, true/false, fill-blank, short answer, essay, constructed response. The current coaching rollout uses the first four as anchors; the last three are stored for a later rollout.

COACHING MODE

Coaching is a standalone product, not an exam setting and not a rewrite of analogy generation. Scope reads Program → Track → Paper. createSession only verifies a qualified anchor and never ships the original stem. Auto notes, five-class error tags, and knowledge-tree lighting are out of MVP.

FROZEN THREE-RUNG CONTRACT

STAGE_01
FULL EXPLANATION

Entry is stage one. Help stays collapsed until asked. A miss retries a variant in the same stage instead of dropping back.

CramAI coaching stage one: worked solution with orientation, method, and worked-step hints
STAGE_02
HALF HINT

A new same-knowledge variant. Stem highlights plus a method hint, not the full worked solution. Stage-2 worked steps open only after two misses.

CramAI coaching stage two: reduced scaffold after worked solution, hints stop at method
STAGE_03
ZERO-HINT CHECK

No help before submit. A zero-hint pass adds a confirmation item. Two consecutive passes write independentPassedAt.

CramAI coaching stage three: independent check with no hints after the first two rungs
PRE-SUBMIT LADDER L1–L4

Each tap opens one more level; levels never drop. There is no skip-to-solution control. Default on for everyone; the adaptive flag is the kill switch.

01

L1 ORIENT

What is being asked. Optional stem highlights. Off in S3.

02

L2 METHOD

Formula name or first path step, with none of this item's numbers. Off in S3.

03

L3 WORKED STEP

Show the first step or drop distractors. In S2, only after two misses. Off in S3.

04

L4 FULL SOLUTION

Only after submit. No pre-submit entry.

MASTERY EVIDENCE

A hinted correct is correct_with_hint, not independent mastery. Mastery = two consecutive zero-hint passes, including a harder confirmation item. A miss on confirmation drops a band, clears the flag, and restarts the pair. A 25% multiple-choice guess still needs confirmation.

WHERE THE NEXT ITEM COMES FROM

The learner gets one item. First fetch uses idempotency key initial:episodeId:stage:1. After submit, prefetch; unused ready cache in the same stage skips generation and billing. Advancing a stage retires old ready items. A variant never becomes the next anchor. A zero-pass batch is not billed.

PRODUCT RULES
  • +Anchors are server-side bank items only; original stems never appear in the learner UI. createSession verifies an anchor and does not ship it.
  • +Generated candidates must keep the anchor type; structure-check failures are dropped. Blind Solver and Judge fail closed; source keys are not the correctness oracle.
  • +Current rollout: single, multiple, true/false, fill-blank. Imported items can be anchors unless explicitly rejected or license-restricted.
  • +Only successfully delivered qualified items are billed; generation failure and empty batches do not charge.
  • +Variant generation, Judge, and adaptive difficulty are owned by teammates on AI services. Everyone else keeps the three-rung contract above.
  • +Coaching must not write ExamAttempt, wrong-answer sets, or knowledge-mastery tables. It uses ExamCoachSession / Episode / Item / Attempt / GenerationJob.
  • +A hinted correct is not mastery. There is no give-up-and-reveal control. Future stars must not reward having seen a solution.
SHIPPED OUTCOME

Mock exams and coaching share one question entity, with separate session domains. Published papers can enter coaching scope. Coaching no longer depends on a throwaway JSON pile, and it does not rewrite the analogy pipeline.

PRODUCT SYSTEMS

05 MODULES
SYS_01

ACCESS & GROWTH LOOP

Moving between beta, open trials, and registration

CramAI English production landing page
LIVE_PRODUCTION // PUBLIC EXPERIENCE ENTRY

KEY DECISION

Do not make the whole page a login wall. Gateway allowlists only the two guest-exam-demo paths. GET withholds answers. A failed grade must not invent a score.

SHIPPED OUTCOME

Guests can finish items, grading, and a report first. guest_exam_funnel is seven fail-open steps. This is a usage path, not a conversion-lift claim.

OPEN DECISION LOG

The product moved through beta-code access, open registration, and research-led gating. I made guest /exam/ a finishable exam: silent Guest, local answers, deterministic grading, a template plan, then the existing register page.

KEY DECISION

Do not make the whole page a login wall. Gateway allowlists only the two guest-exam-demo paths. GET withholds answers. A failed grade must not invent a score.

TRADEOFF

Guest access raises cost and abuse risk, so the cap is 12 items / 32KiB and account-level capabilities stay gated. Phase one never calls AI to write items or grade.

  • +Managed beta-code access, open registration, and research-led gating across product stages
  • +Shipped /exam/ silent Guest, local-only answers, guest_full_loop_12_v2
  • +GET withholds correct answers; POST grades deterministically and keeps browser answers on failure
  • +Seven-step funnel events are fail-open; auth still uses the existing register page
SYS_02

AI TUTOR LEARNING SYSTEM

Bringing the FeaturePanel toolkit into the conversation

CramAI English AI Tutor learning conversation
LIVE_PRODUCTION // AI TUTOR WORKSPACE

KEY DECISION

Do not present generation tools as isolated buttons. Materials enter a Content state machine. Artifacts hang on the same sessionId.

SHIPPED OUTCOME

Users can read, discuss, summarize, test, and review the same material in one flow; FeaturePanel becomes a learning system rather than a toolbox.

OPEN DECISION LOG

FeaturePanel combines summaries, mind maps, flashcards, quizzes, knowledge posters, podcasts, resources, and notes. I set the product mix, interaction model, and priorities, then integrated those outputs into the AI Tutor conversation and active-material context.

KEY DECISION

Do not present generation tools as isolated buttons. Materials enter a Content state machine. Artifacts hang on the same sessionId.

TRADEOFF

A growing toolkit increases cognitive load, requiring shared entry, state, and output patterns plus device-specific material navigation.

  • +Planned and prioritized summaries, mind maps, flashcards, quizzes, posters, podcasts, and more
  • +Hung FeaturePanel artifacts on the same sessionId so the current material is not lost
  • +Rebuilt DocumentViewer multi-file switching and mobile behavior
  • +Fail-closed empty states on parse failure; long documents prefer chapter summaries and document digests
SYS_03

EXAM & ITEM BANK

Papers arrange; questions are reusable entities

CramAI English full-screen mock exam interface
LIVE_PRODUCTION // FULL-SCREEN EXAM

KEY DECISION

Keep mock exams on published papers after audit, preview, apply, and publish — not on demo JSON.

SHIPPED OUTCOME

Answering, grading, weakness review, and follow-up plans run on operational papers.

OPEN DECISION LOG

Demo papers proved the exam UI. Four tables, fail-closed retrieval, and guarded import are in Current Focus. This card is the full-screen exam surface that sits on that bank.

KEY DECISION

Keep mock exams on published papers after audit, preview, apply, and publish — not on demo JSON.

TRADEOFF

A structured bank is slower to stand up than a blob dump; it is what later coaching reuses.

  • +Designed phase-1 tables: RawExamImport → ExamPaper → ExamPaperItem → ExamQuestion
  • +Added Category/Program/Track/template/paper/question + Scope; track retrieval fails closed
  • +Shipped portable import: convert, audit, preview waves, explicit --apply / --publish, Restrict provenance
  • +canonical_hash for identity; answers stored as text arrays; seven types ingested, coaching uses the first four
SYS_04

STAGED COACHING

Standalone domain; original stems stay server-side

CramAI coaching start page with the three-rung contract and live item-bank scope picker
LIVE_PRODUCTION // STANDALONE COACH ENTRY

KEY DECISION

createSession verifies an anchor and never ships it. next-item delivers one item. Mastery needs two consecutive zero-hint passes, including a confirmation item.

SHIPPED OUTCOME

Mock exams and coaching share question entities, with separate session domains. A hinted correct is not mastery.

OPEN DECISION LOG

Coaching is not an exam setting and does not rewrite the analogy API. The screenshots are from live production: pick exam and knowledge point, then walk worked solution, reduced scaffold, and zero-hint check.

KEY DECISION

createSession verifies an anchor and never ships it. next-item delivers one item. Mastery needs two consecutive zero-hint passes, including a confirmation item.

TRADEOFF

Variant generation and Blind Solver / Judge are owned by teammates. I owned the independent tables, state machine, L1–L4, and coaching UI.

  • +Independent ExamCoachSession / Episode / Item / Attempt / GenerationJob; no writes to exam-mistake tables
  • +Three rungs plus L1–L4; a pre-submit hint logs correct_with_hint
  • +createSession never ships original stems; a batch can run six pipelines and the learner only gets delivered
  • +Mastery = two consecutive zero-hint passes, including a harder confirmation item
SYS_05

COMMERCIALIZATION & FEEDBACK

Using GA and Plausible to test product paths

CramAI English subscription plans and commercialization interface
LIVE_PRODUCTION // SUBSCRIPTION VALIDATION

KEY DECISION

Define one deduplicated event contract from landing and guest experience through authentication, activation, and first message, with client/server identity alignment.

SHIPPED OUTCOME

Path events form one continuous journey used to locate drop-off. GA4 numbers are usage signals, not conversion lift.

OPEN DECISION LOG

I contributed commercialization design and owned key event instrumentation, registration funnels, and attribution fixes. The data will not be dressed up as vanity metrics; it is used to find drop-off and decide which product assumption to revisit.

KEY DECISION

Define one deduplicated event contract from landing and guest experience through authentication, activation, and first message, with client/server identity alignment.

TRADEOFF

A shared contract and identity alignment take longer than ad-hoc tracking, but prevent duplicate conversions and broken journeys across login.

  • +Defined the event contract from landing through the seven guest-exam steps to authentication
  • +Aligned client and server identity so the journey does not break across login
  • +Ensured balance refresh failures never lose data and guest sessions never block the main entry

KEY REFLECTIONS

REFLECTION_01

Access is not a one-time switch. Beta codes, open registration, and research-led gating must move with the product stage instead of becoming permanent page logic.

REFLECTION_02

More features do not automatically create value. FeaturePanel only becomes a learning loop when it shares the AI Tutor conversation and material context.

REFLECTION_03

Demo papers validate experience, not content operations. The item bank became the shared base for mock exams and coaching before generated variants were allowed to hang off it.

REFLECTION_04

Data does not need to look good; it needs to be trustworthy. Funnel evidence exists to expose problems and inform the next priority, not to dress up conversion rate.

TECH STACK

INTERFACE
Next.js 15React 19TypeScriptTailwind CSSHeroUI
PRODUCT API
Hono (Node.js)FastAPI (Python)PrismaPostgreSQL
AI SYSTEM
LangGraphLlamaIndexAWS Bedrock
MEASUREMENT
EChartsGA4Plausible