pavel 1ar.ionov
Contact

Note Taking System for Human-Machine Collaboration

Problem of AI context fragmentation across sessions and tools, and how a simple centralized memory system layer fixes it.

On solving long-term memory problem and context fragmentation when working with AI assistants.

There are many approaches that address the problem of memory for AI assistants. Yet most of them are either vendor-locked, or require additional levels of abstractions. The first lockes you into an ecosystem and reduces portability. The second is often brittle.

I wanted to solve the problem when context gets split across sessions, project workspaces, vendors, and tools.

The problems

  1. AI assistants have limited memory. Once you hit a limit, the assistant starts to override it’s own memories. While forgetting sometimes is a bliss, I would like to decide, at least with my helpful machine, what sticks and what not.

  2. Project-based assistant workflows create a routing problem. If you have too little projects, you end up with too much context in each. If you keep separate projects, context files, or instruction bundles inside assistants, you eventually start asking “which project does this belong to?”

  3. If you use more than one assistant, memory splits. Claude knows one set of things. ChatGPT knows another.

  4. Coding assistants are especially isolated. Your coding agent knows about current repository, maybe some generally applicable rules, but misses the business context, recent decisions and bottlenecks.

  5. Assistants are usually passive. They do not automatically check what happened last week, what was recently decided, or what is upcoming. Unless you explicitly feed that context in, they operate on the thin slice visible in the current prompt.

The fix: one memory layer outside the assistant

The solution is to move memory out of the assistant entirely and into a note-taking system that every assistant can read from and write to. The raw notes stay in your main system - daily entries, project documents, whatever you already use. A smaller long-term memory layer gets derived from those notes and kept in a form optimized for fast machine reading.

Three things make this work.

First, every assistant reads from and writes to the same place. That eliminates the fragmentation problem by design. Claude, ChatGPT, a local model, a coding agent - they all pull from one shared context layer.

Second, you maintain a habit of writing down what happened and what is next. The system cannot consolidate what was never captured. This does not need to be elaborate. Timestamped lines during the day are enough: 14:20 shipped the gateway, client approved, 16:45 auth bug was a stale token. The practice is sometimes called interstitial journaling - logging between tasks rather than reconstructing the day from memory at night. You can totally dictate short field notes to any assistant that has access to your daily notes too of course.

Third, the assistant consolidates raw notes into progressively tighter layers from the most granular to the most generalized:

  • Daily notes compress into weekly reviews.
  • Weekly reviews feed durable domain buckets.
  • The buckets produce a short digest that gets loaded at the start of every session.
flowchart TD
    A["Daily Notes\nwhat happened, what is next\n(human)"] --> C["Weekly Reviews\ncompression, signal extraction\n(machine)"]
    B["Project Notes\nactive work, decisions\n(human)"] --> C
    C --> D["LTM Buckets\ndurable domain memory\n(machine)"]
    D --> E["STM Digest\ncurrent state, one line per bucket\n(machine)"]
    E -->|"read first"| F["Any AI Assistant"]
    F -->|"writes back findings"| A

    style A fill:#2f2f35,stroke:#555,color:#f0f0f0
    style B fill:#2f2f35,stroke:#555,color:#f0f0f0
    style C fill:#2f2f35,stroke:#DCA436,color:#f0f0f0
    style D fill:#2f2f35,stroke:#DCA436,color:#f0f0f0
    style E fill:#2f2f35,stroke:#c49028,color:#f0f0f0
    style F fill:#2f2f35,stroke:#c49028,color:#f0f0f0

The agentic retrieval is reversed:

  • Short digest loads in any session.
  • If the task belongs to a specific domain, it opens the relevant bucket.
  • If a deeper timeline matters, it reads the weekly reviews, down to daily notes with corresponding meeting notes and todos.

The capture habit

The system depends on raw capture becoming routine. Daily notes are the input layer. They hold operational trace: what just happened, what changed, what failed, what decision was made. They do not need to be reflective or well-written. They need to be honest and ideally timestamped.

The weekly layer is where noise becomes signal. Once a week, the assistant or the human reviews the previous days and writes a short structured summary: what was planned, what happened, what moved, what stalled, and what the single most important result was. This compression step is the highest-leverage habit in the system. A daily note that says “spent 3 hours on CORS” becomes the weekly line “shipped API gateway, CORS was the last blocker.” That distilled form is what the durable buckets eventually absorb.

Project notes serve a similar function at the scope of a specific engagement. They hold local detail - constraints, open questions, architectural decisions, edge cases - that would be too granular for the daily stream. The important requirement is that these notes do not stay trapped inside a vendor’s project container. The useful parts need to flow back into the shared system during consolidation.

Note: project notes and buckets in the LTM layer are written by AI. That does not mean that you should not write your own project notes and buckets. Think about LTM layer as if that would be an auditor notes - you can read them, but you actually should not override them. It’s the 3rd party perspective on your work. A reflection of what you do.

What the structure looks like

The main notes system stays broad. The derived LTM layer stays deliberately small. Here I exemplify it on folders + .md files basis, but you can adjust it easily with databases, collections, bases, tags or whatever you use inside of your second brain.

notes/
  daily/
  projects/
  ...
  ltm/
    README.md          <- instructions for any AI assistant
    digest.md          <- STM: one line per bucket
    weeks/             <- weekly reviews
    buckets/
      work.md          <- projects, clients, revenue
      tools.md         <- workflows, automation, PKM
      personal.md      <- family, home, life admin
      health.md        <- fitness, habits, wellness
      tech.md          <- stack, errors, environment

The bucket names are yours to choose. Pick three to seven domains that cover the areas of your life where you want continuity. A freelancer might use Work, Clients, Finance, Health, and Tech. A student might use Studies, Side Projects, Health, and Social. The taxonomy matters less than the discipline of using it.

flowchart LR
    LTM["LTM"] --> W["Work\nprojects, clients,\nrevenue"]
    LTM --> T["Tools\nworkflows, automation,\nPKM"]
    LTM --> P["Personal\nfamily, home,\nlife admin"]
    LTM --> H["Health\nfitness, habits,\nwellness"]
    LTM --> TECH["Tech\nstack, errors,\nenvironment"]

    style LTM fill:#2f2f35,stroke:#DCA436,color:#f0f0f0
    style W fill:#2f2f35,stroke:#555,color:#f0f0f0
    style T fill:#2f2f35,stroke:#555,color:#f0f0f0
    style P fill:#2f2f35,stroke:#555,color:#f0f0f0
    style H fill:#2f2f35,stroke:#555,color:#f0f0f0
    style TECH fill:#2f2f35,stroke:#555,color:#f0f0f0

digest.md is the global STM. Every assistant reads it first. If the current task belongs to one area, the assistant then opens the relevant bucket for detail.

- work | API gateway shipped, client B blocked on scope, article draft in progress
- personal | travel next week, admin backlog low
- tech | auth issue solved, stale token pattern documented

The digest is short on purpose. It gives orientation, not exhaustiveness.

Each bucket follows the same internal logic. Recent volatile context at the top, durable knowledge below.

## STM (2026-04-09)

- shipped API gateway for client project | approved, moving to next phase
- invoice automation working | API quirks documented in tech bucket

## Durable

- main client stack: Next.js + Clerk + Stripe | multi-tenant architecture
- consulting model: pay for result preferred over hourly

The STM section gets rewritten every consolidation cycle. The durable section changes only when something structurally shifts. That split gives the assistant both immediacy and stability without forcing it to parse a full archive.

The instruction file

The README.md inside the LTM folder is what turns a collection of files into a memory system. It tells any assistant that opens the folder how to use it: read the digest first, then the relevant buckets, newest entries first, volatile sections at the top, durable sections below. The file does not need to be long.

# Long-Term Memory

Compressed knowledge about my life and work, organized by domain.
Read digest.md first for current state, then relevant buckets.

## How to use this

1. Always read digest.md at session start
2. Read the relevant bucket(s) for the current task
3. Newest entries are at the top of each bucket
4. STM sections are volatile - they change every 1-2 weeks
5. Durable sections are stable - reference them with confidence
6. When you learn something worth keeping, note it - I will route
   it to the right bucket during consolidation

Claude Code picks this up automatically if you name it CLAUDE.md and place it in the LTM directory. ChatGPT reads it if you name it AGENTS.md.

If you are using a cloud-first knowledge storage system (Notion, Craft), you should add to instructions of every AI assistant where the entry point to README.md is. This way it will basically act as CLAUDE/AGENTS.md becasue agents will read it on start, then read the digest, then either reach out to more context or answer your actual question.

Why LTM stays separate from main notes

The separation between the main note system and the LTM layer is not accidental. Raw thoughts stay in daily notes and project documents. The assistant’s compressed interpretation lives in weeks, buckets, and digest files. That prevents a problem that becomes harder to notice the longer you ignore it: once AI starts writing everywhere, the boundary between what you concluded and what was synthesized for you dissolves.

Keeping LTM separate preserves that boundary. You can also mark agent actions explicitly in the raw notes. I use a capture step that writes an entry into daily notes when an agent finishes a task and make assistant tag itself. That keeps the activity visible without collapsing human and machine writing into the same stream.

What this changes

After a few months, the practical difference is continuity. Sessions start closer to the real state of work. There is less re-explaining, less reconstructing, and less repeating of approaches that already failed.

The weekly review habit is valuable even without AI. Forcing yourself to compress a week into structured categories reveals patterns invisible in the daily stream: which projects are actually moving, where time disappears, what keeps getting deferred. AI makes the compressed result usable across tools, but the compression itself is the real work.

Any assistant that reads the digest enters the session with a rough map of current reality instead of a blank page. That is the difference between a tool and a collaborator.

Tooling is secondary

This system works in Craft, Obsidian, Notion. Cloud tools like Notion or Craft make cross-device work easier because the assistant can read and write through an API or MCP integration without additional manual setup. Local tools like Obsidian make ownership robuster - the LTM folder is a directory on your filesystem, and extensions like Obsidian Copilot or a terminal-based coding agent can access it directly.

The choice does not matter much. What matters is that memory does not stay trapped inside the assistant that happened to generate it. If you switch providers, you take five markdown files with you. If you use multiple assistants in parallel, they all read the same layer. The memory belongs to you, not to any product.

If you want help setting up a memory system like this for your team or workflow - 1ar labs can help you design it. We run on this architecture.