Automatic Synchronisation of FTP directory

[Terry Allen]

>On Jul 14, 2007, at 5:42 PM, Terry Allen <hmag at ozemail.com.au> wrote:
>
>>>At 7:49 PM +1000 7/14/07, Terry Allen wrote:
>>>>Hi again,
>>>>    I have a slightly unusual taks to get done as a temporary 
>>>>backup solution for a client - they have 3 separate sites, all 
>>>>with hardware based NAS, which have Windows file sharing & FTP. 
>>>>We need to set up an automated system, with an OSX machine doing 
>>>>a directory sync every day for a few weeks from the FTP server 
>>>>running on the NAS (no, rsync or other options aren't available - 
>>>>must be FTP).
>>>>    Can anyone suggest say a command line I can add as a cronjob, 
>>>>or some open source software which might allow us to do this? It 
>>>>basically needs to log into the FTP server as the 
>>>>username/password, check the list of files (& any changes to 
>>>>those files) & back up those which have changed, then log back 
>>>>out once the job is completed.
>>>>    Any assistance would be appreciated on this.
>>>
>>>What you are describing is not so much a backup as data replication.
>>>
>>>Why must you use FTP if the NAS box has SMB? That would seem to be 
>>>the way to go, and then just use rsync (with all its caveats.)
>>>
>>>Otherwise consider using `find` to locate files that have changed 
>>>or been added, with again all the associated caveats about what 
>>>that means, and ftp these files. You'd just need to keep track of 
>>>when the last time you performed the replication.
>>>
>>>You also don't talk about deleting files that are no longer 
>>>present on the source. You should consider this in your thinking.
>>>--
>>>
>>>-dhan
>>>
>>Hi again,
>>    Yes, data replication is what we need to do & yes, we do need to 
>>delete files which are no longer there (I had thought 
>>synchronisation would cover that, but in any case, yes, that is 
>>true).
>
>Replication is not synchronization. And neither by themselves imply 
>deleting files or handling files that move.
>
>>
>>    The reason we didn't want to use SMB in this case is that the 
>>port on their network for this is not open to the Internet as their 
>>SMB network is otherwise unsecured, though I guess it would be as 
>>simple as directing inward enquiries for that port to the NAS.
>
>"Port on there network"? This suggests that this is behind a 
>firewall on another network. I must have missed this additional 
>criteria in your above request. You'd need to open both the port 
>*and* IP address. Unless this is made further henious by NAPT. In 
>which case FTP is going to be a disaster since it basically doesn't 
>work behind NATP 'firewalls'. FTP belongs at the least on a DMZ for 
>good operations.
>
>>
>>    I'm not sure I understand how I could use rsync for such a 
>>purpose - I thought rsync required an rsync client & server - could 
>>you please elaborate on how we would use rsync for such an exercise 
>>using SMB.
>
>If you don't understand how rsync works then you should read it's 
>man page rather than asking to have is waste our time explaining 
>what it already says.
>
>If FTP is not and option install a FTP based fikesystem, like one 
>from FUSE, and use rysnc (careful of it's metadata caveats).

Hi again,
	My original post mentioned 'synchronisation' - "...with an 
OSX machine doing a directory sync every day for a few weeks from the 
FTP server running on the NAS...", so by definition (or at least in 
my dictionary), that would imply that it would delete the files, so 
my original term was in fact correct for the purpose.
	I didn't think I'd need to spell the entire thing out 
regarding networks, as the earlier post had discussed '3 separate 
sites'. In any case, the networks are all static IPs, not NAPT 
related.
	On a separate note though, I must say I have never ever had 
an issue using FTP servers that I have set up for clients behind NAPT 
routers - in general, it seems to be related to the quality of the 
routers & how they interpret outbound & inbound traffic. Unless the 
user has cheap & nasty equipment, then most routers these days handle 
FTP behind NAPT in an efficient manner using dynamic port allocation 
- I do agree that older equipment did cause much grief in this area.
	Finally, as outlined earlier, I didn't know that rsync could 
be used for anything other than an rsync client/server relationship. 
I'm not sure we could convince the client that opening up their SMB 
network to the Internet in general & the need for then passwording 
the entire SMB network, but I'll see what we can come up with - 
Stephan's mention of lftp seems like it will do exactly what we need, 
so I will further investigate this & also rsync's additional features.
-- 

	Bye for now, Terry Allen 
	___________________________________________________________________
hEARd

Postal Address:
	hEARd, 26B Glenning Rd,	Glenning Valley, NSW 2261, Australia
Internet -
	WWW: http://heard.com.au http://itavservices.com
EMAIL: hmag at ozemail.com.au
Phone: Australia - 02 4388 1400 / International - + 61 2 43881400
Mobile: Australia - 04 28881400 / International - 61 4 28881400
-----------------------------------------------
Non profit promotion for new music - since 1994
-----------------------------------------------