GitHub blocks files larger than 100 MiB. Therefore, if you are migrating a repository with objects larger than 100 Mib to Github, you'll need to migrate those large objects to LFS.
By default, windows and mac do not have a case sensitive filesystem. For this reason, I recommend using linux for lfs migration. Also, if the lfs migration seems to take a long time, this is often due to lots of disk I/O. To speed things up, use a cloud linux instance with max disk I/O.
Make sure you have git-lfs installed.
The first step in migrating to LFS is finding what needs to be migrated. There are several tools that can help you determine if LFS migration is needed, and can help you find the objects that need to be migrated to LFS.
Use git-sizer to determine if LFS migration is needed. Here is a utility script that can be used to run git-sizer on all repos in an org.
Another great tool for understanding blob sizes in a repo is git filter-repo. See these instructions for gathering blob sizing with git filter-repo.
Once you have determined that LFS migration is needed, you'll need a list of file extension that you are going to migrate to LFS. To find this list of extensions for your repository, run the following command from within the repository:
git lfs migrate info --everything --above="100 MB"
This will show you the list of file extension that need to be migrated to LSF in order to push the repository up to Github. You may want to add other extensions to this list based on what you know is stored in the repository. My rule of thumb is: if the file can't be text diffed, it's a good candidate for LFS. Jar and zip files and examples of what I would consider good candidates for LFS. The command above can also be run after migrating large objects to LFS, to ensure that you haven't missed anything that needs to be migrated to LFS.
This would also be a good time to perform any other repo cleanup needed, including removing sensitive data.
Once you have identified large files, migrate them to LFS as follows:
- Checkout all remote branches locally.
for REF in $(git for-each-ref --format='%(refname)' refs/remotes/origin/ | grep -v main | grep -v master | grep -v HEAD); do
BRANCH_NAME=${REF#refs/remotes/origin/}
git branch --track ${BRANCH_NAME} ${REF}
done- Migrate to LFS, specifying the file extensions you want to migrate.
git lfs migrate import --object-map=./lfs-mapping.cvs --everything --include="*.[zZ][iI][pP],*[jJ][aA][rR],*.[dD][mM][gG]"Note: Speficying the file extensions in this way ensures that files with both uppercase and lowercase extensions will be migrated to LFS.
- Push all branches and tags
# Push all branches
for REF in $(git for-each-ref --format='%(refname)' refs/heads); do
BRANCH_NAME=${REF#refs/heads/}
git push -fu origin ${BRANCH_NAME}
done# Push all tags
for REF in $(git for-each-ref --format='%(refname)' refs/tags); do
TAG_NAME=${REF#refs/tags/}
git push -fu origin "$REF":refs/tags/$TAG_NAME
doneAfter migrating large objects to LFS, you want to ensure that future versions of these files are also stored in LFS. To do so run the following command for each extension you want to store in LFS:
git lfs track "*.[zZ][iI][pP]"
git lfs track "*.[jJ][aA][rR]"
git lfs track "*.[dD][mM][gG]"
...Then add and commit the .gitattributes file.
git add .gitattributes
git commit -m "LFS tracking configuration"