'Importing existing files in repo to new LFS storage (using SourceTree and Git LFS)
TL;DR: What is the least intrusive way of migrating existing files to the LFS storage, once LFS has been initialized on the repository, and a suitable .gitattributes file has been prepared but not yet pushed, for a Unity project using Github, Git LFS and SourceTree?
Software/services used:
- Unity 2020 LTS
- Github
- Git LFS, which is part of Github
- SourceTree
Links I'm using:
- Githubs own migration tutorial
- Githubs documentation for the migrate command
- A .gitattributes file, which handles casing-differences in extensions using RegEx (I think).
- Another answered question very much like this one.
- I also took a look at BFG Repo-cleaner, but it seems a bit much for this task.
Motivation for posting
I've been trying to find a guide describing the best way to do this, but most seem to be out-dated or don't include enough information for me to be confident in firing these import-commands on my repo. Others focus a lot on how to set up LFS using a specific server backbone, like Bitbucket Cloud, but I can't find any focusing on Git LFS and SourceTree.
SourceTree has direct integration with Git LFS, but there seems to be no UI-implementation of the migration process for existing files in the repository, so I think I have to rely on the terminal for this part (which I honestly haven't had to use in years, so there's that).
I also have a question about a command for this that I don't see mentioned in my contexted searches: --fixup
The setup
It's not a huge repo; it's a main branch with a few dead branches that don't matter. I just want all the files with certain extensions converted to LFS. My initial thought was to do the migrate-command with --norewrite, because I liked the idea of compacting all the changes into one commit and have no changes to the commit history, but when I realized that means the files would have to stay in the repo for posterity, I changed my mind. I want the repo to shrink, as well, which means I want to replace the files with pointer-files back through the commit history, in order to eliminate the actual files from the repo completely, so they only exist in the LFS-storage.
My questions and options, as I see them
In that other answered question I linked to, a full answer isn't given, but it's very close. Though the answer is very informative, it doesn't answer these questions:
- How to handle case-sensitivity surrounding extensions when writing migrate-commands with --include and --exclude options? Do I need to either go through all existing files in the repo and ensure a similar casing in order to do one-liners, or alternatively fire a separate line for each permutation of the extension I find in the repo? (ignore this question, if --fixup understands the casing-format mentioned in 2. and works for my purposes).
- I see that there is a --fixup option for the migrate command, which isn't mentioned in the search results I'm getting with the given context, except in Githubs documentation for the migrate-command. It says:
"Infer --include and --exclude filters on a per-commit basis based on the .gitattributes files in a repository. In practice, this option imports any filepaths which should be tracked by Git LFS according to the repository's .gitattributes file(s), but aren't already pointers."
Is there some reason why the --fixup option isn't recommended anywhere, when it seems to do exactly what I need? Does it not rewrite the history, or something?
I hope it works, because since --fixup reads the .gitattributes files, it should be able to read the different casing-options from the casing-format in the .gitattributes files. Then I don't have to worry about missing files due to different casings, like when using the --include option.
This is the casing-format used in the .gitattributes file (example for .wav files):
*.[Ww][Aa][Vv]
It works for .gitattributes' normal workings, but doesn't work when you use the format for the --include or --exclude options for the migrate-command, e.g.:
git lfs migrate import --include="*.[Ww][Aa][Vv]"
...does not work. This is what gives the casing-issue mentioned in 1.
I have LFS initialized. I have my .gitattributes file (uncommitted) at the root of the project. So, committing and pushing .gitattributes, and then doing 2. seems to be what I want, right? I should be able to open the terminal in SourceTree, and put in this line:
git lfs migrate import --fixup
or do I need --everything, in order to affect all branches?:
git lfs migrate import --everything --fixup
Also, if --fixup doesn't work, is it just a bunch of these?:
git lfs migrate import --everything --include="*.WAV"
git lfs migrate import --everything --include="*.wav"
...followed by a:
push --force
or
push --force --full
???
That's about where I'm at. I'm not entirely sure which way to go here, or how each of these commands affect the result. I'd appreciate any input on what would be the best course here. If anything, I've tried to be as informative as I can and include the best sources I've found on the subject, to help anyone else asking the same questions.
EDIT: I found this issue at the Git LFS repository, which describes a caveat with the --fixup option; it takes into account when the existing files were added to the repo, and if they were added before the changes to the .gitattributes-file, then they aren't included. So, it is recommended to either rebase, or use filter-branch or filter-repo, in order to add/change the .gitattributes file at the root commit, so that the --fixup option will register the files. The issue deals with some more caveats, like having to uninstall LFS first, but eventually found that solution. This is all just to avoid having to do separate calls for each extension-casing permutation. I don't want to do any of those suggestions, since some seem to run into trouble with them, and I can't get --fixup to work with info, meaning I can't preview what it'll change, so I'm going with --include calls, and just searching up all the current permutations of the extensions of the existing files, and doing --include calls for those. Please, if you do have any good information on the subject, I (and probably many others) would love to hear some easier solutions to this problem.
Thanks in advance and best regards, Jonas Tingmose.
Sources
This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.
Source: Stack Overflow
| Solution | Source |
|---|
