In a Distributed model, each site is self-sustained for the most part. While some connectivity to the primary datacenter is required, the remote site would host its own Email Server, manage its own backups, control its own Internet access, and host its own Shared Files. Application access may still rely on HQ, although many applications support this type of distributed model.
The benefit of a Distributed model is that each site can ‘survive’ on its own. There is no Single Point of Failure in this regard. Also, assuming that the hardware in some of the sites is stored in a secure Server Room, this also would potentially facilitate Business Continuity by utilizing Sites that reference each other as contingency Sites.
When designing distributed systems it is said that the following assumptions should be considered false: 1. The network is reliable. 2. Latency is zero. 3. Bandwidth is infinite. 4. The network is secure. 5. Topology doesn’t change. 6. There is one administrator. 7. Transport cost is zero. 8. The network is homogeneous
By challenging each of these assumptions and looking at the system design within that context it can help identify potential risk areas. Systems that exhibit the key principles, like reliability and availability, have designs that take each of these fallacies into consideration.
When it comes to failures, most fall into one of two buckets: hardware or software related. Hardware failures used to be more common, but with all of the recent innovations in hardware design and manufacturing they tend to be fewer and far between with most of these physical failures tending to be network or drive related. Software failures, on the other hand, come in many more varieties. And software bugs in distributed systems can be difficult to replicate and, consequently, fix. In small, self-Centralized systems it is much easier to simulate the conditions required to replicate and debug issues that appear consistently under a well-defined (but possibly unknown) set of conditions. However, in more complex systems or production environments having many servers, it can be extremely difficult to find and diagnose more unusual bugs. It seems like the bug only appears when you are not looking for it. With more hardware the probability goes up that there will be a failure somewhere. Add more software and the complex interactions between different programs create greater chance for more bugs, including the unusual ones. As a result, any distributed design will carefully consider failure and diagnostic scenarios.
The downside to this approach, obviously, is cost. Not only would this require additional hardware and software costs, but you most certainly would require at least a partial onsite presence at each location regardless of how many remote management components are in place. Another consideration would be the backup architecture. Unless each site had a healthy amount of bandwidth, at least the initial data backup processing would have to be handled locally before being shipped or replicated offsite.
In a purely Centralized model, all computing resources reside at the primary Datacenter. This includes Domain Authentication Services, Email, Applications, and Share Files. Remote Sites would access these resources using Thin Client devices as opposed to PC’s
The benefits of a Centralized model are lower capital and operational cost (minimal hardware at each site), security (all data stored in a secured datacenter), less administrative overhead (fewer resources needed since all equipment is in one location), less backup complexity, and greater control over potential risk areas such as Internet access. The downside to a Centralized model is that your remote site’s WAN connection is now a major point of failure and running your network through a single domain controller makes back-ups essential.
The easiest way to combat the WAN from locking up and failing is to have enhanced monitoring and load balancing. Extensive monitoring and logging is essential to any complex distributed system. Having many services each with a different purpose that are still interacting with one another, can lead to highly unusual bugs. It can be hard to tell where the problem lies and where the issue needs to be resolved. One of the best ways to avoid confusion and help diagnose problems quickly is to be sure that the entire network is monitored. In order to really make the most of the metrics, one must understand what is “normal” for the system. Without a baseline or clear history it is really difficult to determine if something is wrong and should be investigated. For example, problems caused from too much load or traffic on the system would be very hard to diagnose if there was not an easy way to understand what type of load or throughput was typical. Therefore it is not just enough to track metrics, but to also look at trends over time. Detailed logs are useful on a daily bases to identify performance bottlenecks and troubleshooting you network in general but they are never more important that when they are the only clue as to why the network went down in the first place. When this happens the logs will tell you what went wrong but the backups will be the one ultimately save your network. You can have your network backup to the Domain controller and then have the domain controller send the backup of its self to the storage location. It is a good idea to get in the habit of having an external backup location where your backups can be made weekly and safely stored in the case of a disaster.

References: https://www.ibm.com/developerworks/community/blogs/woolf/entry/fallacies_of_distributed_computing6?lang=en