![]() There are also some other "sync"-oriented products that do the last bulletpoint (tolerant of file & folder renames & moves, based on size & content hashes), but are low on my list for other reasons such as Syncthing and Resilio Connect (primarily two-way torrent-based sync), FreeFileSync (GUI), etc.ītrfs and ZFS also can both accomplish this, at least as an end-result. duplicacy, borg, crashplan, carbonite, etc.) do that last point (hash-based comparison), but they are "chunk-based"/checksumming solutions that usually don't produce directly browseable targets. (At least my life.) It can't handle, for example, a simple rename of a folder high in the tree. Rsync can actually do what I'm looking for, with appropriate flags, but only under extremely narrow conditions that rarely happens in real life. (It could be made more efficient with, say, caching of checksums, sizes, and mtimes to local DBs on both sides, and/or caching via xattrs as some other linux file utilities do, but still - it's a far cry better than blindly re-copying everything each time, scanning server files from the client first, or only comparing timestamp and filesize. In that way (assuming no file/folder moves/renames), it's extremely efficient. The entire contents of the target are not required to be pulled over the network so that the client can generate hashes from its contents. If filesizes are the same, rsync looks at the hashes to see if they are bit-for-bit identical, or what parts it needs to send. ![]() With those options in place, the client side scans its file contents and generates hashes, while the server side does the same. Rsync accomplishes this, by connecting a local client invocation of rsync, directly to the specified server's rsync daemon, via SSH. This could cause a routine "mirror", which normally takes seconds to minutes, to suddenly consume several days, just because the user was unfortunate enough to rename a folder high in the tree, with many TBs of data underneath. Do not transfer "new" bits just because a file within the source and target tree changed names or moved to a different folder.Huge limitation of rsync (but not really its purpose anyway in spite of such a feature being requested constantly), which is why I'm investigating a few options, most promisingly rclone:.Does so in an incremental, differential way-only transferring bits that are new.Makes a bit-for-bit mirror of source, on dest.Rclone -a -checksum -delete-after /CLIENT/ hostname:/SERVER/ ![]() What I'm trying to accomplish, is an exact mirror of local "client" to remote (LAN) "server" exactly as would be accomplished by: does it use some kind of local watcher (seems unlikely)? Or by comparing filesize + on both client and server? how it "tracks" renames (and/or moves) e.g. it also "tracks" moves (within the same tree being rclone'd) I did see the reference to the -track-renames flag, but it's very unclear in the documentation (and other references) if: I've read literally every line of the documentation and every page of website and github, and hours of forum posts. Ubuntu 18.04 圆4 Which cloud storage system are you using? (eg Google Drive) (latest) Which OS you are using and how many bits (eg Windows 7, 64 bit) What is your rclone version (output from rclone version) It would seem that this could be one of rclone's differentiating features, but I've only found vague references to renamed files, and references to hashes. Rsync can't really do this, at least not very well, especially for moves or directory renames. There are a few sync utilities that do this (including a couple of bittorrent-based sync utils and a GUI-only utility named "FreeFileSync"). by comparing file content hashes) that the files are the same, and move/rename the old target files/folders to match? Again, seems like another simple question that should have an obvious answer, that I can't seem to find a definitive answer for.Īssuming that the file operations are done within the source tree that rclone is copying, if a source file or folder is renamed and/or moved, are the file or folder contents copied all over again to the target, then the old target locations/files deleted? Or is it smart enough to know (e.g.
0 Comments
Leave a Reply. |