Automatic Synchronisation of FTP directory

[Dan Shoop]

At 8:12 AM +1000 7/16/07, Terry Allen wrote:
>>>Hi again,
>>>    Yes, data replication is what we need to do & yes, we do need 
>>>to delete files which are no longer there (I had thought 
>>>synchronisation would cover that, but in any case, yes, that is 
>>>true).
>>
>>Replication is not synchronization. And neither by themselves imply 
>>deleting files or handling files that move.
>>
>>>
>>>    The reason we didn't want to use SMB in this case is that the 
>>>port on their network for this is not open to the Internet as 
>>>their SMB network is otherwise unsecured, though I guess it would 
>>>be as simple as directing inward enquiries for that port to the 
>>>NAS.
>>
>>"Port on there network"? This suggests that this is behind a 
>>firewall on another network. I must have missed this additional 
>>criteria in your above request. You'd need to open both the port 
>>*and* IP address. Unless this is made further henious by NAPT. In 
>>which case FTP is going to be a disaster since it basically doesn't 
>>work behind NATP 'firewalls'. FTP belongs at the least on a DMZ for 
>>good operations.
>>
>>>
>>>    I'm not sure I understand how I could use rsync for such a 
>>>purpose - I thought rsync required an rsync client & server - 
>>>could you please elaborate on how we would use rsync for such an 
>>>exercise using SMB.
>>
>>If you don't understand how rsync works then you should read it's 
>>man page rather than asking to have is waste our time explaining 
>>what it already says.
>>
>>If FTP is not and option install a FTP based fikesystem, like one 
>>from FUSE, and use rysnc (careful of it's metadata caveats).
>
>Hi again,
>	My original post mentioned 'synchronisation' - "...with an 
>OSX machine doing a directory sync every day for a few weeks from 
>the FTP server running on the NAS...", so by definition (or at least 
>in my dictionary), that would imply that it would delete the files, 
>so my original term was in fact correct for the purpose.

Actually it is far from clear. You could "sync" new files, of just 
data in existing files, or sync exisiting files. Removing files is 
normally regarded as a different case. As is what to do with moved 
files. It's a matter of how you treat a files on the target as 
opposed to what you do in terms of sync with source files.

>	I didn't think I'd need to spell the entire thing out 
>regarding networks, as the earlier post had discussed '3 separate 
>sites'. In any case, the networks are all static IPs, not NAPT 
>related.

Well it is important, as it will factor into what one would recommend.

"Sites" is an ambiguous word as it could mean "web site", "system", 
or facility location.

>	On a separate note though, I must say I have never ever had 
>an issue using FTP servers that I have set up for clients behind 
>NAPT routers - in general, it seems to be related to the quality of 
>the routers & how they interpret outbound & inbound traffic.

FTP was never designed for use through NAPT or NAT. As such it's 
obscure way of handling connections is extremely problematic.

>  Unless the user has cheap & nasty equipment, then most routers 
>these days handle FTP behind NAPT in an efficient manner using 
>dynamic port allocation

No the issues are the same regardless of the equipment. And the 
better equipment tends to at least also at stateful inspection which 
further aggravates the process.

Basically if you try and open ports based on state you get nowhere fast.

The best solution if you must do FTP behind NAT/NAPT is a FTP proxy, 
yet few firewalls provide these.

The best overall solution is to place the FTP server on a DMZ where 
it can live on the Internet like FTP intended.

>	Finally, as outlined earlier, I didn't know that rsync could 
>be used for anything other than an rsync client/server relationship.

Noted. However any googling these days should have suggested otherwise.

>  I'm not sure we could convince the client that opening up their SMB 
>network to the Internet in general

No one implied this was necessary.

>  & the need for then passwording the entire SMB network,

No one said this was necessary (but it is how SMB traditionally works.)

>  but I'll see what we can come up with - Stephan's mention of lftp 
>seems like it will do exactly what we need, so I will further 
>investigate this & also rsync's additional features.

And offers all the problems of FTP.

Best of luck.
-- 

-dhan

------------------------------------------------------------------------
Dan Shoop                                                   AIM: iWiring
Systems & Networks Architect                      http://www.ustsvs.com/
shoop at iwiring.net                                http://www.iwiring.net/
1-714-363-1174