CI
CircleCI Pipeline Optimization
Caching, parallelism, resource classes, Docker layer caching, and build speed strategies
Details
Language / Topic
CircleCICategory
Performance
Rules
balanced
- Cache dependencies with checksum-based keys tied to lockfiles: `key: deps-v1-{{ checksum "package-lock.json" }}`
- Use parallelism with test splitting to distribute tests across containers: `parallelism: 4` with `circleci tests split --split-by=timings`
- Choose the right resource_class per job: small for linting, medium for unit tests, large/xlarge for builds and E2E
- Enable Docker layer caching (DLC) on setup_remote_docker to speed up Docker image builds
- Use workspace persistence (persist_to_workspace/attach_workspace) to share build artifacts between jobs without rebuilding
- Cache dependencies with precise keys: `key: deps-v1-{{ checksum "package-lock.json" }}` — use a version prefix (v1, v2) to manually invalidate when needed
- Use fallback cache keys for partial matches: `keys: [deps-v1-{{ checksum "lock" }}, deps-v1-]` — stale cache is faster than cold install
- Set parallelism on test jobs and split by timing data: `parallelism: 4` with `circleci tests split --split-by=timings` — timing data from store_test_results enables even distribution
- Right-size resource_class: small (1 vCPU) for lint/format checks, medium (2 vCPU) for unit tests, large (4 vCPU) for heavy builds, xlarge for E2E suites
- Enable Docker layer caching on setup_remote_docker: `docker_layer_caching: true` — caches individual Docker layers across builds for faster image builds
- Use persist_to_workspace to share build output between jobs: build once in the build job, attach workspace in deploy/test jobs
- Run independent jobs in parallel via workflows: lint, unit-test, security-scan concurrently — build job requires all three
- Use path-based filtering with dynamic config to skip irrelevant jobs: docs-only changes skip the full test suite
- Use CircleCI convenience images (cimg/*) for faster container startup — smaller, pre-configured, regularly updated
- Store test results with store_test_results for timing-based test splitting and test insights in the CircleCI dashboard
- Set no_output_timeout appropriately: increase for long-running builds, decrease to fail fast on stuck steps