Automatic Synchronisation of FTP directory

[Dan Shoop]

On Jul 14, 2007, at 5:42 PM, Terry Allen <hmag at ozemail.com.au> wrote:

>> At 7:49 PM +1000 7/14/07, Terry Allen wrote:
>>> Hi again,
>>>    I have a slightly unusual taks to get done as a temporary  
>>> backup solution for a client - they have 3 separate sites, all  
>>> with hardware based NAS, which have Windows file sharing & FTP. We  
>>> need to set up an automated system, with an OSX machine doing a  
>>> directory sync every day for a few weeks from the FTP server  
>>> running on the NAS (no, rsync or other options aren't available -  
>>> must be FTP).
>>>    Can anyone suggest say a command line I can add as a cronjob,  
>>> or some open source software which might allow us to do this? It  
>>> basically needs to log into the FTP server as the username/ 
>>> password, check the list of files (& any changes to those files) &  
>>> back up those which have changed, then log back out once the job  
>>> is completed.
>>>    Any assistance would be appreciated on this.
>>
>> What you are describing is not so much a backup as data replication.
>>
>> Why must you use FTP if the NAS box has SMB? That would seem to be  
>> the way to go, and then just use rsync (with all its caveats.)
>>
>> Otherwise consider using `find` to locate files that have changed  
>> or been added, with again all the associated caveats about what  
>> that means, and ftp these files. You'd just need to keep track of  
>> when the last time you performed the replication.
>>
>> You also don't talk about deleting files that are no longer present  
>> on the source. You should consider this in your thinking.
>> --
>>
>> -dhan
>>
> Hi again,
>    Yes, data replication is what we need to do & yes, we do need to  
> delete files which are no longer there (I had thought  
> synchronisation would cover that, but in any case, yes, that is true).

Replication is not synchronization. And neither by themselves imply  
deleting files or handling files that move.

>
>    The reason we didn't want to use SMB in this case is that the  
> port on their network for this is not open to the Internet as their  
> SMB network is otherwise unsecured, though I guess it would be as  
> simple as directing inward enquiries for that port to the NAS.

"Port on there network"? This suggests that this is behind a firewall  
on another network. I must have missed this additional criteria in  
your above request. You'd need to open both the port *and* IP address.  
Unless this is made further henious by NAPT. In which case FTP is  
going to be a disaster since it basically doesn't work behind NATP  
'firewalls'. FTP belongs at the least on a DMZ for good operations.

>
>    I'm not sure I understand how I could use rsync for such a  
> purpose - I thought rsync required an rsync client & server - could  
> you please elaborate on how we would use rsync for such an exercise  
> using SMB.

If you don't understand how rsync works then you should read it's man  
page rather than asking to have is waste our time explaining  what it  
already says.

If FTP is not and option install a FTP based fikesystem, like one from  
FUSE, and use rysnc (careful of it's metadata caveats).