GIT-LFS-MIGRATE(1)

πŸ‘‘ GIT-LFS-MIGRATE(1) πŸ‘‘ β€” like, the git-lfs-migrate manual page, but make it make sense πŸ’…

okay bestie, buckle up πŸŽ€ this is the git-lfs-migrate page from the Git LFS fam, and she is a lot but also so worth it. let's get into it ✨


✨ NAME ✨

git-lfs-migrate β€” migrate history to or from Git LFS πŸ¦‹


✨ SYNOPSIS ✨

git lfs migrate <mode> [options] [--] [branch ...]

✨ DESCRIPTION ✨

okay so here's the vibe: git-lfs-migrate converts files in a Git repo to or from Git LFS pointers, or it summarizes Git file sizes by file type. the import mode converts Git files (i.e., blobs) to Git LFS, while the export mode does the reverse πŸ’« and the info mode gives you an informational summary which is lowkey useful for deciding which files to import or export πŸ‘‘

in all modes, by default git lfs migrate operates only on the currently checked-out branch, and only on files (of any size and type, no cap) added in commits which do not exist on any remote. multiple options are available to override these defaults, so don't stress 🫢

when converting files to or from Git LFS, the git lfs migrate command will only make changes to your local repository and working copy, never any remotes. this is intentional fr β€” the import and export modes are generally "destructive" in the sense that they rewrite your Git history, changing commits and generating new commit SHAs 😭 (the exception is the "no-rewrite" import sub-mode; see IMPORT WITHOUT REWRITING HISTORY for details).

so like, you should always first commit or stash any uncommitted work before using the import or export modes πŸ’€ and then validate the result of the migration before pushing the changes to your remotes β€” for instance by running the info mode and by examining your rewritten commit history. yes bestie, both things.

once you are satisfied with the changes, you will need to force-push the new Git history of any rewritten branches to all your remotes πŸ”₯ this is a step which should be taken with care, since you will be altering the Git history on your remotes. dramatic but true.

to examine or modify files in branches other than the currently checked-out one, branch refs may be specified directly, or provided in one or more --include-ref options. they may also be excluded by prefixing them with ^ or providing them in --exclude-ref options. use the --everything option to specify that all refs should be examined, including all remote refs. see INCLUDE AND EXCLUDE REFERENCES for details πŸ‘€

for the info and import modes, all file types are considered by default; while useful in the info mode, this is often not desirable when importing, so either filename patterns (pathspecs) or the --fixup option should normally be specified in that case. (at least one include pathspec is required for the export mode 🚨) pathspecs may be defined using the --include and --exclude options (-I and -X for short), as described in INCLUDE AND EXCLUDE.

as typical Git LFS usage depends on tracking specific file types using filename patterns defined in .gitattributes files, the git lfs migrate command will examine, create, and modify .gitattributes files as necessary. the .gitattributes files will always be assigned the default read/write permissions mode (i.e., without execute permissions). any symbolic links with that name will cause the migration to halt prematurely πŸ™ˆ

the import mode (see IMPORT) will convert Git objects of the file types specified (e.g., with --include) to Git LFS pointers, and will add entries for those file types to .gitattributes files, creating those files if they do not exist. the result should be as if git lfs track commands had been run at the points in your Git history corresponding to where each type of converted file first appears ✨ the exception is if the --fixup option is given, in which case the import mode will only examine any existing .gitattributes files and then convert Git objects which should be tracked by Git LFS but are not yet.

the export mode (see EXPORT) works as the reverse operation to the import mode, converting any Git LFS pointers that match the file types specified with --include, which must be given at least once. note that .gitattributes entries will not be removed, nor will the files; instead, the export mode inserts "do not track" entries similar to those created by the git lfs untrack command. the --remote option is available in the export mode to specify the remote from which Git LFS objects should be fetched if they do not exist in the local Git LFS object cache; if not provided, origin is used by default 🧚

the info mode (see INFO) summarizes by file type (i.e., by filename extension) the total number and size of files in a repository. note that like the other two modes, by default the info mode operates only on the currently checked-out branch and only on commits which do not exist on any remote, so to get a summary of the entire repository across all branches, use the --everything option. if objects have already been converted to Git LFS pointers, then by default the size of the referenced objects is totaled and reported separately πŸ’– you may also choose to ignore them by using --pointers=ignore or to treat the pointers as files by using --pointers=no-follow. (the latter option is akin to how existing Git LFS pointers were handled by the info mode in prior versions of Git LFS).

when using the --everything option, take note that it means all commits reachable from all refs (local and remote) will be considered, but not necessarily all file types πŸ‘€ the import and info modes consider all file types by default, although the --include and --exclude options constrain this behavior.

while the --everything option means all commits reachable from any ref will be considered for migration, after migration only local refs will be updated even when --everything is specified 🚨 this ensures remote refs stay synchronized with their remote. in other words, refs/heads/foo will be updated with the --everything option, but refs/remotes/origin/foo will not, so it stays in sync with the remote until git push origin foo is performed. after checking that the results of a migration with --everything are satisfactory, it may be convenient to push all local branches to your remotes by using the --all option to git push πŸ’…

unless the --skip-fetch option is given, git lfs migrate always begins by fetching updated lists of refs from all the remotes returned by git remote, but as noted above, after making changes to your local Git history while converting objects, it will never automatically push those changes to your remotes. periodt. πŸ”₯


🌸 MODES 🌸

  • info β€” show information about repository size. see INFO πŸ’–
  • import β€” convert Git objects to Git LFS pointers. see IMPORT and IMPORT WITHOUT REWRITING HISTORY ✨
  • export β€” convert Git LFS pointers to Git objects. see EXPORT πŸ¦‹

πŸ’… OPTIONS πŸ’…

  • -I <paths>, --include=<paths> β€” bestie, this one's all about which paths you're pulling in. Full tea in INCLUDE AND EXCLUDE below πŸ‘‡
  • -X <paths>, --exclude=<paths> β€” and this is the opposite, paths you wanna skip. Also in INCLUDE AND EXCLUDE ✨
  • --include-ref=<refname> β€” pull in commits reachable from this ref. Deets in INCLUDE AND EXCLUDE REFERENCES 🫢
  • --exclude-ref=<refname> β€” nope out of commits reachable from this ref. Same section, iykyk πŸ’œ
  • --skip-fetch β€” okay so this one says "trust me, I already know the full set of remote refs, don't go refresh them" when it's figuring out which "un-pushed" commits to migrate. Heads up tho: it does literally nothing if you also pass --include-ref or --exclude-ref πŸ™ˆ
  • --everything β€” migrate everything, all the refs. Again, see INCLUDE AND EXCLUDE REFERENCES πŸ’–
⚠️ Note: Git refs are "case-sensitive" on all platforms in "packed from" (peep git-pack-refs(1)). So on "case-insensitive" file systems β€” like NTFS on Windows, or default APFS on macOS β€” git-lfs-migrate(1) would only migrate the first ref if two or more refs are equal except for upper/lower case letters. Lowkey a trap, no cap πŸ’€
  • --yes β€” auto-yes to any prompts, so you can run it noninteractively. Right now the only such prompt is the one asking whether to overwrite (destroy) any working copy changes. Which means… specifying this option may cause data loss if you are not careful 😭 please be careful bestie
  • [branch ...] β€” migrates only the set of branches you list. Don't list any? Then git-lfs-migrate(1) just does the currently checked out branch, simple as that 🌸
    • References starting with ^ get excluded, while branches that don't start with ^ get included.
    • If any of --include-ref or --exclude-ref are given, the checked out branch will not be appended β€” but branches given explicitly still get appended ✨

πŸŽ€ INFO πŸŽ€

The info mode is the vibe-check mode: it summarizes the sizes of file objects chillin' in the Git history. It supports all the core migrate options plus these extra ones:

  • --above=<size> β€” only count files whose individual filesize is above the given size. The size can be a number of bytes, or a number followed by a storage unit, e.g. "1b", "20 MB", "3 TiB", etc. πŸ“
    • If a set of files sharing a common extension has no files in that set whose individual size is above the given --above, then no entry for that set will be shown. (Like, the whole set ghosts the output πŸ‘»)
  • --top=<n> β€” only display the top n entries, ordered by how many total files match the given pathspec. Default is to show only the top 5 entries. When existing Git LFS objects are found, an extra, separate "LFS Objects" line is output in addition to the top n entries β€” unless you use the --pointers option to change this behavior πŸ’…
  • --unit=<unit> β€” format the number of bytes in each entry as a quantity of the storage unit you pick. Valid units include:
    • b, kib, mib, gib, tib, pib β€” for IEC storage units
    • b, kb, mb, gb, tb, pb β€” for SI storage units

If a --unit is not specified, the largest unit that can fit the number of counted bytes as a whole number quantity is chosen. She picks for you, iconic πŸ‘‘

  • --pointers=[follow|no-follow|ignore] β€” treat existing Git LFS pointers in the history according to one of three alternatives:
    • In the default follow case, if any pointers are found, an additional separate "LFS Objects" line item is output which summarizes the total number and size of the Git LFS objects referenced by pointers.
    • In the ignore case, any pointers are simply ignored.
    • While the no-follow case replicates the behavior of the info mode in older Git LFS versions and treats any pointers it finds as if they were regular files β€” so the output totals only include the contents of the pointers, not the contents of the objects to which they refer. πŸ¦‹
  • --fixup β€” infer --include and --exclude filters on a per-commit basis based on the .gitattributes files in a repository. In practice, this option counts any filepaths which should be tracked by Git LFS according to the repository's .gitattributes file(s), but aren't already pointers. The .gitattributes files are not reported, in contrast to the normal output of the info mode. 🚨 This option is incompatible with explicitly given --include, --exclude filters and with any --pointers setting other than ignore β€” hence --fixup implies --pointers=ignore if it is not explicitly set.

The format of the output shows the filename pattern, the total size of the file objects (excluding those below the --above threshold, if one was defined), and the ratio of the number of files above the threshold to the total number of files; this ratio is also shown as a percentage. For example:

    *.gif               93 MB   9480/10504 files(s)  90%
    *.png               14 MB    1732/1877 files(s)  92%

By default only the top five entries are shown, but --top allows for more or fewer to be output as desired. Slay 🌈


πŸ”₯ IMPORT πŸ”₯

The import mode migrates objects present in the Git history to pointer files tracked and stored with Git LFS. It supports all the core migrate options and these additional ones:

  • --verbose β€” print the commit oid and filename of migrated files to STDOUT. So you can watch the chaos in real time πŸ‘€
  • --above=<size> β€” only migrate files whose individual filesize is above the given size. size may be a number of bytes, or a number followed by a storage unit, e.g. "1b", "20 MB", "3 TiB", etc. 🚫 This option cannot be used with the --include, --exclude, and --fixup options. Pick a lane bestie
  • --object-map=<path> β€” write to path a file with the mapping of each rewritten commits. The file format is CSV with this pattern: OLD-SHA,NEW-SHA πŸ“
  • --no-rewrite β€” migrate objects to Git LFS in a new commit without rewriting Git history. Heads up tho: when this option is used, the migrate import command will expect a different argument list, specialized options will become available, and the core migrate options will be ignored. See IMPORT WITHOUT REWRITING HISTORY πŸ’–
  • --fixup β€” infer --include and --exclude filters on a per-commit basis based on the .gitattributes files in a repository. In practice, this option imports any filepaths which should be tracked by Git LFS according to the repository's .gitattributes file(s), but aren't already pointers. 🚨 This option is incompatible with explicitly given --include, --exclude filters.

If --no-rewrite is not provided and --include or --exclude (-I, -X, respectively) are given, the .gitattributes will be modified to include any new filepath patterns as given by those flags. ✨

If --no-rewrite is not provided and neither of those flags are given, the gitattributes will be incrementally modified to include new filepath extensions as they are rewritten in history. It just… keeps adding, okay? πŸ’…


🧚 IMPORT WITHOUT REWRITING HISTORY 🧚

The import mode has a special sub-mode enabled by the --no-rewrite flag. This sub-mode will migrate objects to pointers as in the base import mode, but will do so in a new commit without rewriting Git history. When using this sub-mode, the base migrate options β€” such as --include-ref β€” will be ignored, as will those for the base import mode. The migrate command will also take a different argument list. As a result of these changes, --no-rewrite will only operate on the current branch β€” any other interested branches must have the generated commit merged in. Main character branch energy only 😌

The --no-rewrite sub-mode supports the following options and arguments:

  • -m <message>, --message=<message> β€” specifies a commit message for the newly created commit ✍️
  • [file ...] β€” the list of files to import. These files must be tracked by patterns specified in the gitattributes. No sneaky untracked files, k? πŸ™ˆ

If --message is given, the new commit will be created with the provided message. If no message is given, a commit message will be generated based on the file arguments. She'll write it for you, cute πŸŽ€


πŸ’œ EXPORT πŸ’œ

The export mode migrates Git LFS pointer files present in the Git history out of Git LFS, converting them into their corresponding object files. It supports all the core migrate options and these additional ones:

  • --verbose β€” print the commit oid and filename of migrated files to STDOUT 🫢
  • --object-map=<path> β€” write to path a file with the mapping of each rewritten commit. The file format is CSV with this pattern: OLD-SHA,NEW-SHA πŸ“Š
  • --remote=<git-remote> β€” download LFS objects from the provided git-remote during the export. If not provided, defaults to origin. Classic 🌸

The export mode requires at minimum a pattern provided with the --include argument to specify which files to export. Files matching the --include patterns will be removed from Git LFS, while files matching the --exclude patterns will retain their Git LFS status. The export command will modify the .gitattributes to set/unset any filepath patterns as given by those flags. Balance restored βš–οΈ


🌸 INCLUDE AND EXCLUDE 🌸

You can specify that git lfs migrate should only convert files whose pathspec matches the --include glob patterns and does not match the --exclude glob patterns β€” either to reduce total migration time or to only migrate part of your repo. Multiple patterns may be given using commas as delimiters. Comma-separated, periodt ✨

Pattern matching is done so as to be functionally equivalent to the pattern matching format of .gitattributes. In addition to simple file extension matches (e.g. .gif) patterns may also specify directory paths, in which case the path/* format may be used to match recursively. πŸ¦‹

πŸ“Œ Note that this form of pattern matching for the --include and --exclude options used by the git lfs migrate command is unique among the suite of git lfs commands. Other commands which also take these options, such as git lfs ls-files, use the gitignore(5) form of pattern matching instead. Don't get it twisted bestie πŸ’€

🌈 INCLUDE AND EXCLUDE REFERENCES 🌈

You can specify that git lfs migrate should only convert files added in commits reachable from certain references, namely those defined using one or more --include-ref options, and should ignore files in commits reachable from references defined in --exclude-ref options. Selective, we love to see it πŸ‘‘

                   D---E---F
                  /         \
             A---B------C    refs/heads/my-feature
              \          \
               \          refs/heads/main
                \
                 refs/remotes/origin/main

In the above configuration, the following commits are reachable by each ref:

  • refs/heads/main: C, B, A
  • refs/heads/my-feature: F, E, D, B, A
  • refs/remote/origin/main: A

The following git lfs migrate options would, therefore, include commits F, E, D, C, and B, but exclude commit A:

      --include-ref=refs/heads/my-feature
      --include-ref=refs/heads/main
      --exclude-ref=refs/remotes/origin/main

The presence of flag --everything indicates that all commits reachable from all local and remote references should be migrated (but note that the remote refs themselves will not be updated). Like, the refs stay exactly where they are, okay? Don't @ me πŸ’…

🌸 EXAMPLES 🌸

πŸ’Ύ Migrate unpushed commits

A super common use case for migrate is converting big Git objects into LFS before you push your commits. By default, it only scans commits that don’t exist on any remote, so long as the repository is non-bare. No cap.

First, run git lfs migrate info to list the file types taking up the most space in your repository:

$ git lfs migrate info
Fetching remote refs: ..., done
Sorting commits: ..., done
Examining commits: 100% (1/1), done
*.mp3   284 MB    1/1 files(s)  100%
*.pdf   42 MB     8/8 files(s)  100%
*.psd   9.8 MB  15/15 files(s)  100%
*.ipynb 6.9 MB    6/6 files(s)  100%
*.csv   5.8 MB    2/2 files(s)  100%

Now you can run git lfs migrate import to convert some file types to LFS:

$ git lfs migrate import --include="*.mp3,*.psd"
Fetching remote refs: ..., done
Sorting commits: ..., done
Rewriting commits: 100% (1/1), done
  main  d2b959babd099fe70da1c1512e2475e8a24de163 -> 136e706bf1ae79643915c134e17a6c933fd53c61
Updating refs: ..., done

If after conversion you find that some files in your working directory have been replaced with Git LFS pointers, that’s normal, bestie β€” no panic. The working copies of these files can be repopulated with their full expected contents by using git lfs checkout.

🌿 Migrate local history

You can also migrate the entire history of your repository:

# Check for large files and existing Git LFS objects in your local main branch
$ git lfs migrate info --include-ref=main

# Check for large files and existing Git LFS objects in every branch
$ git lfs migrate info --everything

# Check for large files in every branch, ignoring any existing Git LFS objects,
# and listing the top 100 or fewer results
$ git lfs migrate info --everything --pointers=ignore --top=100

The same flags will work in import mode:

# Convert all zip files in your main branch
$ git lfs migrate import --include-ref=main --include="*.zip"

# Convert all zip files in every local branch
$ git lfs migrate import --everything --include="*.zip"

# Convert all files over 100K in every local branch
$ git lfs migrate import --everything --above=100Kb
⚠️ Note: This will require a force-push to any existing Git remotes. Using the --all option when force-pushing may be convenient if many local refs were updated, e.g., after importing to Git LFS with the --everything option.

✨ Migrate without rewriting local history (using migrate import --no-rewrite) ✨

You can also migrate files without modifying the existing history of your repository. Iconic, honestly.

Migrating to Git LFS in a single new commit, without specifying a commit message:

$ git lfs track "*.zip" "*.mp3" "*.psd"
$ git add .gitattributes
$ git commit -m "add Git LFS attributes"

$ git lfs migrate import --no-rewrite --yes test.zip audios/*.mp3 images/*.psd

Note that the .mp3 and .psd filename patterns are not quoted and so will be expanded by the shell, not by the git lfs migrate import command. See the remarks following the next example for further details.

Migrating to Git LFS in a single new commit, with a specified commit message:

$ git lfs track "*.zip" "*.mp3" "*.psd"
$ git add .gitattributes
$ git commit -m "add Git LFS attributes"

$ git lfs migrate import --no-rewrite --yes \
  -m "Import test.zip, audios/*.mp3, and images/*.psd files" \
  test.zip audios/*.mp3 images/*.psd

Note that the .mp3 and .psd filename patterns are not quoted and so will be expanded by the shell, not by the git lfs migrate import command. Using quotation marks around such patterns with the --no-rewrite option is not supported, as the command expects a list of files (possibly as expanded by the shell) and will not search for all files matching a quoted filename pattern.

Additional commands may be used to find and list all files matching a set of patterns, of course. In a POSIX-compliant environment such as Linux, macOS, or Git Bash on Windows, the example above could be combined with the find(1) and sed(1) commands as follows, so long as the resulting list of files does not cause the length of the expanded command line to exceed the maximum permitted by the shell:

$ git lfs migrate import --no-rewrite --yes \
  $(find . -name "*.zip" -o -name "*.mp3" -o -name "*.psd" | sed "s/^\.\///")

Also note that the git lfs track command will by default change the modification timestamps on any existing files which match the given filename patterns, so the files will appear "dirty" to Git and thereby prompt the user to convert them to Git LFS files. The subsequent git lfs migrate import command will then caution that the working tree contains modifications and ask the user for confirmation to proceed unless the --yes option is specified, as in the examples above.

🧚 Migrate without rewriting local history (using add --renormalize)

As an alternative to using git lfs migrate import with the --no-rewrite option to migrate files to Git LFS without modifying the existing history of your repository, if you have Git v2.16.0 or higher installed, you can use the --renormalize option of the git add command to achieve the same result. For example:

$ git lfs track "*.zip" "*.mp3" "*.psd"
$ git add .gitattributes
$ git add --renormalize .
$ git commit -m "Import all *.zip, *.mp3, and *.psd files to Git LFS"

πŸ¦‹ Migrate from Git LFS

If you no longer wish to use Git LFS for some or all of your files, you can use the export mode to convert Git LFS objects into regular Git blobs again. Slay.

The export mode requires at least one --include pathspec, and will download any objects not found locally from your origin Git remote, or from the Git remote you specify with the --remote option.

# Convert all zip Git LFS objects to files in your main branch
$ git lfs migrate export --include-ref=main --include="*.zip"

# Convert all zip Git LFS objects to files in every local branch,
# fetching any object data not cached locally from the my-remote Git remote
$ git lfs migrate export --everything --include="*.zip" --remote=my-remote

# Convert all Git LFS objects to files in every local branch
$ git lfs migrate export --everything --include="*"
⚠️ Note: This will require a force-push to any existing Git remotes. Using the --all option when force-pushing may be convenient if many local refs were updated, e.g., after exporting from Git LFS with the --everything option.

πŸ‘€ SEE ALSO πŸ‘€

Part of the git-lfs(1) suite.


And that’s the whole page, bestie πŸ’– Footer/source-version-date check: GIT-LFS-MIGRATE(1) β€” exactly the footer energy, no invented extras, periodt. Part of the git-lfs(1) suite. Bye! πŸ«ΆπŸŽ€

πŸ“¦ where this page comes from