Opt-in experiments

The Git project tests new performance features behind feature.experimental. Enabling it activates settings under evaluation for default-on. This is the place to look when tracking the bleeding edge of Git performance.

What it currently sets

Across recent versions, feature.experimental=true has at various times included:

The bundle setting

Tuning monorepos involves a dozen settings. Rather than memorize and apply each, Git provides "feature flags" that activate a coherent bundle. feature.manyFiles is the most useful: one knob that enables all the index and stat optimizations.

Index versions

Git's index has had several formats. Version 2 is the historic baseline. Version 3 added skip-worktree and other extensions. Version 4 (recommended for large repos) adds prefix compression of paths, halving index size on deep trees. Version 4 became default in Git 2.37+ for new repos via feature.manyFiles.

Setting

git config index.version 4
git update-index --index-version 4

The first sets the policy; the second rewrites the existing index file in the new format. Run both for an existing repo.

The built-in daemon

Git 2.36 introduced a built-in fsmonitor daemon, eliminating the need for external services like Watchman. It runs as git fsmonitor--daemon, listens to OS file events, and answers Git's queries with the list of changed paths since a token timestamp.

The status walk problem

git status compares working tree to index, walking every tracked file's stat info. On a million-file repo this can take seconds. Fsmonitor uses the operating system's file change notifications (FSEvents on macOS, ReadDirectoryChangesW on Windows, inotify on Linux) to know which files changed since the last query, skipping the rest.

What Scalar is

Scalar (originally a Microsoft tool, bundled with Git in 2.38+) is a wrapper that applies the full performance stack — partial clone, sparse checkout, sparse index, fsmonitor, commit-graph, MIDX, geometric repack, background maintenance — with one command. It is the recommended starting point for cloning a large repository.

The filter language

Partial clone filters are specified by short DSL strings. Each filter excludes objects matching the spec from the transfer; missing objects are fetched on demand from the promisor remote.

Lazy object fetching

Partial clone (--filter) lets a clone omit some objects from the initial transfer, fetching them lazily when needed. The promisor remote keeps its promise to provide them on demand. This is the foundation of usable monorepo clones — full history, fraction of the data.

Why the index is slow

Even with sparse checkout, the index by default contains an entry per file in the entire repo — a million entries on a monorepo, even if only a thousand are present in the working tree. Reading and writing this on every command is expensive. The sparse index (Git 2.37+) replaces excluded directories with a single tree entry, shrinking the index by 100x or more.

The monorepo working set

In a monorepo, individual developers usually touch a small subset — one or two apps, a handful of libraries. Sparse checkout populates only those paths in the working tree, leaving the rest as skip-worktree entries in the index. The result: faster git status, smaller IDE indexes, less disk.