I have a few big projects at work that I’ve created over the years. An inventory application, status management, and of course all the scripts (some 200 of them).
Initially I’d work on the inventory and status apps on the live server. Changes were minor and new features could be added without disruption. I used RCS, the simple revision control system that comes with Unix and that I’ve used extensively in the past to manage DNS files. But as time went on, some of the updates might take a couple of days to get working which impacted the application. As a result, I took the two desktops I’d salvaged, installed Red Hat and Ubuntu on the two, and copied the source code (php scripts) over to the server as a simple location to work on the scripts.
About 2 years back, I went though a major revision to the Inventory and started looking at creating an actual development environment where there’s a central store of ready to use code and a separate code directory where I could work on stuff without impacting the STABLE code base. It took a bit of work and several changes to make things work cleanly (and of course I’m still updating things).
With the virtualization I now have at home, I’m busily moving that code base over to the new environment. The nice thing is all the test bits I’ve done over the years can be moved over to the test location and the actual sites can be pared down to just the necessary bits. Cleaner and it protects the site a bit.
Environment now:
code – This is a directory with all the code for the web sites. I have a good 15 sites at home and 5 or 6 sites at work.
archive – Old stuff I don’t need but want to hold on to for historical references.
static – Any non-source code bits. Images for the most part but some data files that are imported regularly.
stage – The staging area for the code. The site is assembled into this directory and then sync’d with the production server.
html – The working web directory. All work is done here.
In the code site directory are the following utility scripts:
findcount – This script runs the find -print command to generate a list of all the files in the source which is stored in the countall file.
fixsettings – This script recreates the link to all the settings files to ensure every file has the same settings information. This script is in the html working area.
searchall – This script lets you search all the scripts for a string (pass -i to ignore case). This script is also in the html working area.
In the code directory are two files for each site, three if you include the log file.
make[site] – This is a script that builds the site located in the staging area.
– Runs the findcount script and compares it to the countall.backup file. If a new script has been added, this reminds you to add it to the manifest file.
– Parses the manifest file and creates directories and installs files as listed.
– Uses rsync to copy any static files into the staging area.
– Compares the manifest output with the countall output files to list any scripts that were added but failed to get added to the manifest file.
– A flag is created indicating the site has been updated and needs to be sync’d with the production site.
manifest.[site] – List of directories that need to be created and files that are copied from the source code directory into the staging area.
In the staging area are also three files, four if you count the log file.
sync[site] – This script rsync’s the site to all the target production sites. For work, I have the Inventory going to three servers right now due to transitioning to a new server.
sync.[site] – This is the flag file created by the make[site] script in the code directory. The sync[site] script only sync’s with the target server when this file exists.
exclude.[site] – This is a list of files and/or directories to exclude from the sync process.
For my account, I have two sets of lines in cron. The first set runs at 1am and runs the make[site] script for each listed site. This lets me automatically update the sites each night even if there are no changes (rsync is set with –delete so files that shouldn’t be there are deleted). The second set of lines run the sync[site] commands every minute. At 1am when the make[site] scripts run, they create the sync.[site] flag files and a minute later the sync[site] script runs and updates the site. But also, when I make a change to a site and manually run the make[site] script, the sync[site] script runs after a minute and updates the site.
Other scripts I have are the ”’check”’ scripts.
checkout – Uses the co -l command to check out a script from the source code into the working directory.
checkin – Uses the ci -u command to check in a script.
checkrw – Checks the source code site to see which files have been checked out and are being worked on.
checkmanifest – Checks working site against the manifest and provides a difference between any checked out scripts and STABLE scripts.
checkdiff – Runs a diff between a specific checked out script and the STATIC script.
checkinventory – Tells you what files are in the working directory that aren’t in the site directory.
And an import script that retrieves the mysql database from the target server and imports the data into mysql.
And that’s the environment. It works pretty well for what I’m trying to accomplish. I would like to use git to manage sites but I haven’t been able to find a good tutorial on how to pull files for sites.