git clone git lfs 出现错误 fatal: model-00001-of-00002.safetensors: smudge filter lfs failed

开源 0

git clone git lfs 出现错误 fatal: model-00001-of-00002.safetensors: smudge filter lfs failed
error: external filter ‘git-lfs filter-process’ failed
fatal: model-00001-of-00002.safetensors: smudge filter lfs failed。

Error downloading object: model-00001-of-00002.safetensors (66dec18): Smudge error: Error downloading model-00001-of-00002.safetensors (66dec18c9f1705b9387d62f8485f4e7d871ca388718786737ed3c72dbfaac9fb): batch response: Repository or object not found: https://huggingface.co/meta-llama/Llama-2-7b-chat-hf.git/info/lfs/objects/batch
Check that it exists and that you have proper access to it

Errors logged to /root/meta-llama/Llama-2-7b-chat-hf/.git/lfs/logs/20240221T024656.481462682.logUse `git lfs logs last` to view the log.error: external filter 'git-lfs filter-process' failedfatal: model-00001-of-00002.safetensors: smudge filter lfs failedwarning: Clone succeeded, but checkout failed.You can inspect what was checked out with 'git status'and retry with 'git restore --source=HEAD :/'

详细错误如下：

git-lfs/2.11.0 (GitHub; linux amd64; go 1.13.4)
git version 2.27.0

$ git-lfs filter-process
Error downloading object: model-00001-of-00002.safetensors (66dec18): Smudge error: Error downloading model-00001-of-00002.safetensors (66dec18c9f1705b9387d62f8485f4e7d871ca388718786737ed3c72dbfaac9fb): batch response: Repository or object not found: https://huggingface.co/meta-llama/Llama-2-7b-chat-hf.git/info/lfs/objects/batch
Check that it exists and that you have proper access to it

Repository or object not found: https://huggingface.co/meta-llama/Llama-2-7b-chat-hf.git/info/lfs/objects/batch
Check that it exists and that you have proper access to it
github.com/git-lfs/git-lfs/errors.Errorf
/builddir/build/BUILD/git-lfs-2.11.0/_build/src/github.com/git-lfs/git-lfs/errors/errors.go:69
github.com/git-lfs/git-lfs/lfshttp.defaultError
/builddir/build/BUILD/git-lfs-2.11.0/_build/src/github.com/git-lfs/git-lfs/lfshttp/errors.go:128
github.com/git-lfs/git-lfs/lfshttp.(*Client).handleResponse
/builddir/build/BUILD/git-lfs-2.11.0/_build/src/github.com/git-lfs/git-lfs/lfshttp/errors.go:51
github.com/git-lfs/git-lfs/lfshttp.(*Client).DoWithRedirect
/builddir/build/BUILD/git-lfs-2.11.0/_build/src/github.com/git-lfs/git-lfs/lfshttp/client.go:326
github.com/git-lfs/git-lfs/lfsapi.(*Client).doWithCreds
/builddir/build/BUILD/git-lfs-2.11.0/_build/src/github.com/git-lfs/git-lfs/lfsapi/auth.go:103
github.com/git-lfs/git-lfs/lfsapi.(*Client).doWithAuth
/builddir/build/BUILD/git-lfs-2.11.0/_build/src/github.com/git-lfs/git-lfs/lfsapi/auth.go:67
github.com/git-lfs/git-lfs/lfsapi.(*Client).DoWithAuth
/builddir/build/BUILD/git-lfs-2.11.0/_build/src/github.com/git-lfs/git-lfs/lfsapi/auth.go:25
github.com/git-lfs/git-lfs/lfsapi.(*Client).DoAPIRequestWithAuth
/builddir/build/BUILD/git-lfs-2.11.0/_build/src/github.com/git-lfs/git-lfs/lfsapi/auth.go:56
github.com/git-lfs/git-lfs/tq.(*tqClient).Batch
/builddir/build/BUILD/git-lfs-2.11.0/_build/src/github.com/git-lfs/git-lfs/tq/api.go:74
github.com/git-lfs/git-lfs/tq.Batch
/builddir/build/BUILD/git-lfs-2.11.0/_build/src/github.com/git-lfs/git-lfs/tq/api.go:40
github.com/git-lfs/git-lfs/tq.(*TransferQueue).enqueueAndCollectRetriesFor
/builddir/build/BUILD/git-lfs-2.11.0/_build/src/github.com/git-lfs/git-lfs/tq/transfer_queue.go:559
github.com/git-lfs/git-lfs/tq.(*TransferQueue).collectBatches.func1
/builddir/build/BUILD/git-lfs-2.11.0/_build/src/github.com/git-lfs/git-lfs/tq/transfer_queue.go:453
runtime.goexit
/usr/lib/golang/src/runtime/asm_amd64.s:1357
batch response
github.com/git-lfs/git-lfs/errors.newWrappedError
/builddir/build/BUILD/git-lfs-2.11.0/_build/src/github.com/git-lfs/git-lfs/errors/types.go:198
github.com/git-lfs/git-lfs/errors.Wrap
/builddir/build/BUILD/git-lfs-2.11.0/_build/src/github.com/git-lfs/git-lfs/errors/errors.go:74
github.com/git-lfs/git-lfs/tq.(*tqClient).Batch
/builddir/build/BUILD/git-lfs-2.11.0/_build/src/github.com/git-lfs/git-lfs/tq/api.go:77
github.com/git-lfs/git-lfs/tq.Batch
/builddir/build/BUILD/git-lfs-2.11.0/_build/src/github.com/git-lfs/git-lfs/tq/api.go:40
github.com/git-lfs/git-lfs/tq.(*TransferQueue).enqueueAndCollectRetriesFor
/builddir/build/BUILD/git-lfs-2.11.0/_build/src/github.com/git-lfs/git-lfs/tq/transfer_queue.go:559
github.com/git-lfs/git-lfs/tq.(*TransferQueue).collectBatches.func1
/builddir/build/BUILD/git-lfs-2.11.0/_build/src/github.com/git-lfs/git-lfs/tq/transfer_queue.go:453
runtime.goexit
/usr/lib/golang/src/runtime/asm_amd64.s:1357
Error downloading model-00001-of-00002.safetensors (66dec18c9f1705b9387d62f8485f4e7d871ca388718786737ed3c72dbfaac9fb)
github.com/git-lfs/git-lfs/errors.newWrappedError
/builddir/build/BUILD/git-lfs-2.11.0/_build/src/github.com/git-lfs/git-lfs/errors/types.go:198
github.com/git-lfs/git-lfs/errors.Wrapf
/builddir/build/BUILD/git-lfs-2.11.0/_build/src/github.com/git-lfs/git-lfs/errors/errors.go:85
github.com/git-lfs/git-lfs/lfs.(*GitFilter).downloadFile
/builddir/build/BUILD/git-lfs-2.11.0/_build/src/github.com/git-lfs/git-lfs/lfs/gitfilter_smudge.go:115
github.com/git-lfs/git-lfs/lfs.(*GitFilter).Smudge
/builddir/build/BUILD/git-lfs-2.11.0/_build/src/github.com/git-lfs/git-lfs/lfs/gitfilter_smudge.go:76
github.com/git-lfs/git-lfs/commands.smudge
/builddir/build/BUILD/git-lfs-2.11.0/_build/src/github.com/git-lfs/git-lfs/commands/command_smudge.go:127
github.com/git-lfs/git-lfs/commands.filterCommand
/builddir/build/BUILD/git-lfs-2.11.0/_build/src/github.com/git-lfs/git-lfs/commands/command_filter_process.go:120
github.com/git-lfs/git-lfs/vendor/github.com/spf13/cobra.(*Command).execute
/builddir/build/BUILD/git-lfs-2.11.0/_build/src/github.com/git-lfs/git-lfs/vendor/github.com/spf13/cobra/command.go:766
github.com/git-lfs/git-lfs/vendor/github.com/spf13/cobra.(*Command).ExecuteC
/builddir/build/BUILD/git-lfs-2.11.0/_build/src/github.com/git-lfs/git-lfs/vendor/github.com/spf13/cobra/command.go:852
github.com/git-lfs/git-lfs/vendor/github.com/spf13/cobra.(*Command).Execute
/builddir/build/BUILD/git-lfs-2.11.0/_build/src/github.com/git-lfs/git-lfs/vendor/github.com/spf13/cobra/command.go:800
github.com/git-lfs/git-lfs/commands.Run
/builddir/build/BUILD/git-lfs-2.11.0/_build/src/github.com/git-lfs/git-lfs/commands/run.go:103
main.main
/builddir/build/BUILD/git-lfs-2.11.0/_build/src/github.com/git-lfs/git-lfs/git-lfs.go:33
runtime.main
/usr/lib/golang/src/runtime/proc.go:203
runtime.goexit
/usr/lib/golang/src/runtime/asm_amd64.s:1357
Smudge error
github.com/git-lfs/git-lfs/errors.newWrappedError
/builddir/build/BUILD/git-lfs-2.11.0/_build/src/github.com/git-lfs/git-lfs/errors/types.go:198
github.com/git-lfs/git-lfs/errors.NewSmudgeError
/builddir/build/BUILD/git-lfs-2.11.0/_build/src/github.com/git-lfs/git-lfs/errors/types.go:284
github.com/git-lfs/git-lfs/lfs.(*GitFilter).Smudge
/builddir/build/BUILD/git-lfs-2.11.0/_build/src/github.com/git-lfs/git-lfs/lfs/gitfilter_smudge.go:85
github.com/git-lfs/git-lfs/commands.smudge
/builddir/build/BUILD/git-lfs-2.11.0/_build/src/github.com/git-lfs/git-lfs/commands/command_smudge.go:127
github.com/git-lfs/git-lfs/commands.filterCommand
/builddir/build/BUILD/git-lfs-2.11.0/_build/src/github.com/git-lfs/git-lfs/commands/command_filter_process.go:120
github.com/git-lfs/git-lfs/vendor/github.com/spf13/cobra.(*Command).execute
/builddir/build/BUILD/git-lfs-2.11.0/_build/src/github.com/git-lfs/git-lfs/vendor/github.com/spf13/cobra/command.go:766
github.com/git-lfs/git-lfs/vendor/github.com/spf13/cobra.(*Command).ExecuteC
/builddir/build/BUILD/git-lfs-2.11.0/_build/src/github.com/git-lfs/git-lfs/vendor/github.com/spf13/cobra/command.go:852
github.com/git-lfs/git-lfs/vendor/github.com/spf13/cobra.(*Command).Execute
/builddir/build/BUILD/git-lfs-2.11.0/_build/src/github.com/git-lfs/git-lfs/vendor/github.com/spf13/cobra/command.go:800
github.com/git-lfs/git-lfs/commands.Run
/builddir/build/BUILD/git-lfs-2.11.0/_build/src/github.com/git-lfs/git-lfs/commands/run.go:103
main.main
/builddir/build/BUILD/git-lfs-2.11.0/_build/src/github.com/git-lfs/git-lfs/git-lfs.go:33
runtime.main
/usr/lib/golang/src/runtime/proc.go:203
runtime.goexit
/usr/lib/golang/src/runtime/asm_amd64.s:1357

Current time in UTC:
2024-02-21 02:46:56

ENV:
LocalWorkingDir=/root/meta-llama/Llama-2-7b-chat-hf
LocalGitDir=/root/meta-llama/Llama-2-7b-chat-hf/.git
LocalGitStorageDir=/root/meta-llama/Llama-2-7b-chat-hf/.git
LocalMediaDir=/root/meta-llama/Llama-2-7b-chat-hf/.git/lfs/objects
LocalReferenceDirs=
TempDir=/root/meta-llama/Llama-2-7b-chat-hf/.git/lfs/tmp
ConcurrentTransfers=8
TusTransfers=false
BasicTransfersOnly=false
SkipDownloadErrors=false
FetchRecentAlways=false
FetchRecentRefsDays=7
FetchRecentCommitsDays=0
FetchRecentRefsIncludeRemotes=true
PruneOffsetDays=3
PruneVerifyRemoteAlways=false
PruneRemoteName=origin
LfsStorageDir=/root/meta-llama/Llama-2-7b-chat-hf/.git/lfs
AccessDownload=none
AccessUpload=none
DownloadTransfers=basic,lfs-standalone-file
UploadTransfers=basic,lfs-standalone-file
GIT_DIR=/root/meta-llama/Llama-2-7b-chat-hf/.git
GIT_EXEC_PATH=/usr/libexec/git-core

Client IP addresses:
172.17.0.2

解决方法：

// Skip smudge - We'll download binary files later in a faster batchgit lfs install --skip-smudge// Do git clone heregit clone ...// Fetch all the binary files in the new clonecd projectgit lfs pull// Reinstate smudgegit lfs install --force

如果是国外的大文件，比如 大模型，最好配置git的代理，如下：

git config --global http.sslVerify "false"git config --global advice.detachedHead false#建议socks5 和http 都配置一下代理#配置socks5#git config --global http.proxy socks5 127.0.0.1:7890#git config --global https.proxy socks5 127.0.0.1:789#配置http#git config --global http.proxy 127.0.0.1:7890#git config --global https.proxy 127.0.0.1:7890#查看设置：#git config --global --get http.proxy#git config --global --get https.proxy#取消办法：#git config --global --unset http.proxy#git config --global --unset https.proxy

就可以正常下载了：

Downloading LFS objects:  20% (1/5), 298 MB | 3.1 MB/s

也许您对下面的内容还感兴趣：