Overview

This document is written as an aid for AgroClimate and FAWN personnel in migrating their web applications to a new location within OSG's hosting offerings.

Background

Over the past three years the AgroClimate and FAWN projects have utilized physical servers co-located in the CNS SSRB Datacenter and managed by the CNS Open Systems Group. In May 2009 a proposal to migrate away from this old, unsupported infrastructure to current OSG hosting infrastructure was agreed upon. This document will address the steps that must be taken to move forward on this migration.

Old Infrastructure

In order to understand the migration we will start by understanding the existing, old infrastructure and configuration.

Hardware

The hardware is comprised of two IBM x346 servers running RedHat 5.2. These servers are configured to host both the AgroClimate website and database and the FAWN website and database and also provide shell access, cron, R statistical software, and anonymous FTP upload of datasets.

Websites

The current production websites live on the /nerdc/www/ NFS share at /nerdc/www/www.agclimate.org/ and /nerdc/www/fawn.ifas.ufl.edu/ respectively.

Databases

Each project has its own MySQL instance that runs on each physical server. AgroClimate's DB instance runs on port 3306 and FAWN's DB instance runs on port 3307. Currently the database instances on if-fwn-prdw01 are acting as the primary/master instance with the database instances on if-fwn-prdd01 acting as a slave/standby instance.

New Infrastructure

The infrastructure to replace the old IBM servers is based on more-or-less standard OSG hosting offerings- Apache web hosting with php fast-cgi, MySQL, and Vmware virtual machine resources. There are two differences, however, in that (1) OSG will provide fully managed hosting for Agro/FAWN's VM and (2) AgroClimate's MySQL presence will be on dedicated instances of MySQL.

Summary of services

• 1 prod and 1 test website for AgroClimate ( 2 x Apache hosting)
• 1 prod and 1 test MySQL instance for AgroClimate ( 2 x MySQL instance hosting [special])
• 1 prod website for FAWN ( 1 x Apache hosting )
• 1 prod MySQL DB for FAWN ( 1 x MySQL DB hosting [standard] )
• 1 Standard VM hosting unit
• Fully managed hosting for 1 machine

Virtual Machine

A virtual machine that will be managed by CNS/OSG has been created and installed for use by the AgroClimate and FAWN projects. This system will primarily be used to run cron jobs, dataset uploads, anonymous ftp downloads for fawn, R statistical analysis, and general shell access for the projects' developers.

The hostname of this VM is agrofawn-prod01.osg.ufl.edu

The VM has been configured to allow Gatorlink login via SSH by the Ag/FAWN developers. Additionally this server will accept logins from the kerberos identities that were created as “service users” for AgroClimate and FAWN. The cnswww-* users will also be the file owners of the respective website contents.

• cnswww-agcl : this user is the shared “service” user for AgroClimate
• cnswww-fawn : this user is the shared “service” user for FAWN
• agro-georgia : this user is a shared “service” user for the AgroClimate associates in Georgia

Websites

The new websites will live in the same location as the old websites: on the /nerdc/www/ NFS share at /nerdc/www/www.agclimate.org/ and /nerdc/www/fawn.ifas.ufl.edu/ respectively.

Databases

FAWNs database will reside in our shared MySQL instance. This resource has already been created within our shared production MySQL environment- the DB name is fawn.

AgroClimate has a proliferation of databases within MySQL that they maintain in support of their application and its accompanying tools. To accommodate this we will provide them with two dedicated MySQL instances for their test and production environments. Because this is a non-standard offering a special price of $60/month/instance was agreed upon.

The production and test AgroClimate MySQL instances have already been created:

• AgroClimate prod DB inst.: mysql.osg.ufl.edu:3310
• AgroClimate test DB inst.: mysql.osg.ufl.edu:3311 (currently called 3311fawn on the servers)
• FAWN prod DB: mysql.osg.ufl.edu:3306:fawn

phpMyAdmin is available for all of these MySQL resources. This web interface is available at:

https://open-systems.ufl.edu/phpMyAdmin/<port>/

where <port> is the port number of the relevant MySQL database instance. Any access requires authentication with a valid Gatorlink credential followed by phpMyAdmin login using local MySQL credentials.

Migration Process

Pre-testing

1. Create the test websites:
   For purposes of testing the Agro and FAWN web applications these websites have been created on OSG's TEST web cluster: agroclimate-test.osg.ufl.edu and fawn-test.osg.ufl.edu. Copy production document root to the new test document root:
   /nerdc/www/www.agclimate.org/ to /nerdc/www/agroclimate-test.osg.ufl.edu
   /nerdc/www/fawn.ifas.ufl.edu/ to /nerdc/www/fawn-test.osg.ufl.edu
2. Dump the existing database and load the data to the new database.
3. Update database references in the new test document root to point to the new database:
   agroclimate: mysql.osg.ufl.edu:3310
   fawn: mysql.osg.ufl.edu:3306:fawn
4. Agro and FAWN will need to create cron jobs and other background processes on the agrofawn-prod01.osg.ufl.edu server.  These will replace the cronjobs that currently live on if-fwn-prdd01 and if-fwn-prdw01.

Testing

Prior to AgroClimate's website redesign a year or two ago, the web app's framework depended on php's “register globals” setting being enabled; we are not sure if this is still true. This option is not enabled in the new environment and is a possible cause for migration problems.

Migration

1. Websites: When testing has been completed the contents of the successfully tested document root should be brought online on OSG's production web cluster. This will need to be done in conjunction with DNS support from Agclimate and IFAS to update the DNS entries for www.agroclimate.org and fawn.ifas.ufl.edu. These names should be moved to point at a dispatched, shared IP that is served by the production cluster. These sites do not need SSL, so this should simplify the configuration.
2. Databases: dump the existing databases and shut them down. Upload data to the new databases. Make sure that any code referencing the old databases, either to upload or download data, now references the new databases.
3. Php-fastcgi: Both projects' web applications will depend upon php-fastcgi within our websvc resources. Currently, the agroclimate-test and fawn-test websites are backed by test websvc infrastructure. As moves to production occur new websvc/php-fastcgi resources will need to be configured on OSG's production websvc servers (this is done by CNS OSG).

Notes

The existing test server, elnino.agen.ufl.edu, will also no longer be supported by CNS OSG.  Instead all testing should occur on the test resources provisioned in OSG's hosting infrastructure and the agrofawn-prod01.osg.ufl.edu VM.

Other Resource information (glossary)

Hostnames and IPs for OLD Systems:

Server Hostnames / IPs:

• if-fwn-prdw01.osg.ufl.edu / 128.227.74.214
• if-fwn-prdd01.osg.ufl.edu / 128.227.74.215

Dispatched Hostnames / Dedicated IPs: *

• virtual-agclimate-web1.osg.ufl.edu / 128.227.74.224
• virtual-fawn-web1.osg.ufl.edu / 128.227.74.225

Non-dispatched Hostnames / Dedicated IPs: **

• virtual-ifas-db1.osg.ufl.edu / 128.227.74.226
• virtual-ifas-db2.osg.ufl.edu / 128.227.74.62

Other DNS names, not controlled by CNS:

• (www.)agroclimate.org – an A-record run by the AgroClimate folks that points at 128.227.74.214
• fawn.ifas.ufl.edu – a CNAME run by the FAWN/IFAS folks that points at virtual-fawn-web1.osg.ufl.edu

NOTES:
* The virtual-<project>-web1 name/IP is the dispatched ip for each project's website.
** The virtual-ifas-db[12] names/IPs are manually moved between the servers as a consistent point of access to the MySQL databases- this was done to help ensure write consistency for data loads, etc.

Hostnames and IPs for NEW Systems:

Server Hostnames / IPs:

• agrofawn-prod01.osg.ufl.edu / 128.227.74.17

Test websites to use during migration:

• agroclimate-test.osg.ufl.edu
• fawn-test.osg.ufl.edu

Database resources:

• mysql.osg.ufl.edu:3306:fawn FAWN production MySQL DB
• mysql.osg.ufl.edu:3310  AgroClimate production MySQL instance
• mysql.osg.ufl.edu:3311  AgroClimate test MySQL instance