Measure the Size, Then Choose a Strategy
The first step when CI slows is quantification. Use `git count-objects -vH` for object store size, `git rev-list --count HEAD` for commit count, and `du -sh .git` for disk usage. Past roughly 1GB or tens of thousands of commits, the cost stops being network transfer and becomes pack inflation plus index building.
Locate the bloat with `git verify-pack -v .git/objects/pack/*.idx | sort -k3 -n -r | head -20`, then map objects to paths with `git rev-list --objects --all`. Almost every "the repo is too big" case comes down to a handful of causes: accidentally committed binaries (video, model weights, build output), long-lived unpruned branches, or vendored dependencies.
Remember history only grows: even if the current tree is 1MB, a 2GB tar committed two years ago still has to be downloaded during clone. That is why cleanup requires `git filter-repo` history rewriting plus a force push, not merely `git rm`.
# 量化仓库体积
git count-objects -vH
git rev-list --count HEAD
du -sh .git
# 找出最大的历史对象及其路径
git verify-pack -v .git/objects/pack/*.idx | sort -k3 -n -r | head -20
git rev-list --objects --all | git cat-file --batch-check="%(objecttype) %(objectname) %(objectsize) %(rest)" \
| awk "$1==\"blob\" {print $3, $4}" | sort -n -r | head -20
# 清理历史大文件（会改写所有提交 sha，需强推）
git filter-repo --path build/ --path "*.zip" --invert-paths
git push --force --mirror originThe Real Boundaries of Shallow and Partial Clone
`--depth 1` fetches only the tip commit, turning a multi-minute clone into seconds — at the cost of breaking history operations. `git log` shows nothing meaningful, `git blame` reports missing commits, and `git describe --tags` finds no tags. Recover on demand with `git fetch --deepen=100`, or `git fetch --unshallow` when you truly need everything.
`--filter=blob:none` is a partial clone: it fetches commits and trees but pulls blob contents lazily from the remote. It shines on repositories whose history holds large binaries, because checking out the current branch transparently fetches just the blobs it needs. The cost is one network round-trip per lazy fetch, which can be slower on poor connections.
The two combine, subject to server support: `--filter` requires `uploadpack.allowFilter` on the remote, and `--depth` with partial clone has compatibility gaps on older servers. A safe combination is `--depth 50 --filter=blob:none --no-tags`, which keeps blame functional while avoiding a full historical blob transfer.
# 日常 CI 克隆：够用且可 blame
git clone --depth 50 --filter=blob:none --no-tags --branch "$CI_BRANCH" \
"https://github.com/org/repo.git" .
# 需要完整历史或 tag 时按需加深
git fetch --deepen=200
git fetch --tags
# 明确放弃历史（纯构建场景）
git clone --depth 1 --single-branch --no-tags "https://github.com/org/repo.git" .
# 提交历史很大时用 sparse-checkout 只取需要的目录
git sparse-checkout init --cone
git sparse-checkout set packages/api packages/sharedCache Key Design: Hit Rate Versus Correctness
A CI cache key trades hit rate against correctness. A common mistake is putting the branch name in the key, which gives every branch and every PR its own cache and wrecks the hit rate. A more workable design is layered: a primary key of lockfile hash plus runtime version plus architecture, with the branch name as a prefix and a prefix fallback lookup.
Fallback lookups via `restore-keys` drive the hit rate. Restoring a slightly older cache under a prefix like `node_modules-` and then running `npm ci` to repair it is far faster than a cold install. Over-precise keys — the full lockfile hash — mean a single dependency tweak sends the whole pipeline back to cold.
Another trap to avoid is caching `node_modules` without reinstalling on restore. Native modules tied to a platform or runtime version get silently corrupted across versions. The reliable pattern is caching the package manager download cache (npm `_cacache`, the pnpm store) and re-running installation every time.
# GitHub Actions：主键精确 + 前缀回退
- name: Cache pnpm store
uses: actions/cache@v4
with:
path: ~/.pnpm-store
key: pnpm-${{ runner.os }}-${{ runner.arch }}-node${{ hashFiles("pnpm-lock.yaml") }}
restore-keys: |
pnpm-${{ runner.os }}-${{ runner.arch }}-node
# GitLab CI：分阶段缓存，同样避免键过窄
stages: [deps, test]
cache:
key:
files:
- pnpm-lock.yaml
prefix: "pnpm-$CI_OS"
paths:
- .pnpm-store/
- node_modules/
deps:
script:
- corepack enable
- pnpm install --frozen-lockfile
test:
script:
- pnpm testSeparate Artifacts from Source
Mixing build output with source is where cache design starts to rot. The correct structure is: the source repository holds source only, CI produces immutable artifacts (image tarballs, tar.gz, jar, wheel), those go to an artifact registry (GHCR, Nexus, S3), and deployment pulls by immutable tag without ever checking out source.
This separation buys three things directly: rollback to any historical tag because artifacts are immutable; the build runs once per source change while deployment becomes a pure pull; and production needs no compiler toolchain, allowing small images with a minimal attack surface.
Enforce one rule: deployment may only consume signed artifacts produced by CI, and build commands are forbidden in the deploy stage. The moment deployment builds locally, artifacts stop being reproducible and both the caching and rollback benefits evaporate.
# CI：构建并推送不可变制品（tag 含 git sha，天然不可变）
docker build -t ghcr.io/org/app:$GIT_SHA -t ghcr.io/org/app:latest .
docker push ghcr.io/org/app:$GIT_SHA
echo "artifact=ghcr.io/org/app:$GIT_SHA" >> $GITHUB_OUTPUT
# 部署：只按 tag 拉取制品，不构建、不 checkout 源码
git checkout $TARGET_SHA
git pull --ff-only
docker pull ghcr.io/org/app:$ARTIFACT_TAG
docker tag ghcr.io/org/app:$ARTIFACT_TAG ghcr.io/org/app:running
docker compose up -d --no-build
# 回滚：换成旧 sha tag 重新拉取即可，无需重新构建
# ARTIFACT_TAG=<previous-sha> ./deploy.sh