Difference between revisions of "ProjectInfrastructure"

From RackTables Wiki
Jump to navigation Jump to search
(→‎git mirrors (code): git.racktables.org replaces code.racktables.org)
 
(19 intermediate revisions by 2 users not shown)
Line 1: Line 1:
 
==Mapping of assets==
 
==Mapping of assets==
 
=== DNS ===
 
=== DNS ===
−
racktables.org domain is currently registered through GoDaddy, which also provides 2 free nameservers.
+
racktables.org domain is currently registered through GoDaddy, which also provides 2 free nameservers. Aaron manages the domain.
 +
 
 
=== racktables.org mail ===
 
=== racktables.org mail ===
−
"racktables.org mail is handled by 10 mx.yandex.ru." Yandex Mail is a free email service, which allows for an own DNS domain. There is a single "devteam" mailbox configured in the domain, which is setup to forward all messages to every development team member.
+
racktables.org mail is handled by zohomail.com. Zoho Mail offers free service for up to three e-mail addresses. Two mailing lists exist, "devteam" and "info", which are setup to forward all messages to every development team member.
 +
 
 
=== racktables-users mail ===
 
=== racktables-users mail ===
−
FreeLists is a great free service without accompanying advertisements. The only drawback about it is that it is non-profit and can go down some day, so '''make sure to have the list of users on the backup list'''.
+
FreeLists is a great free service without accompanying advertisements. The only drawback about it is that it is non-profit and can go down some day, hence '''make sure the list of subscribers is present in a recent backup tarball'''.
−
=== Demo host ===
+
 
−
Arnaud Launay is the administrator of a server, which currently runs RackTables demo. To get SSH access there, be a team member and find someone who already has access.
+
To manage the mailing list, go to [http://www.freelists.org/login.html this page] and use the form to the left. Input your email subscribed to the list. If you don't remember your password, log in with an empty password, in this case the system will email you a one-time token to input on the next screen. Once logged in, you will see a very long SELECT with all lists of FreeLists. Click on the SELECT and press "r" to get to items starting with R. Select racktables-users and press "Select list" below. On the next screen press "Admin Menu". To manage individual subscribers, select one on the "Current List Users" SELECT and press "User Options". VACATION, MODPOST and NOPOST are the most frequently used flags, their meaning is explained inline.
 +
 
 +
Please think twice before changing any of the list-wide settings on the "Edit List Config" menu, these settings influence all subscribers of racktables-users.
 +
 
 +
=== Most web-sites (demo, bugs, wiki, www) ===
 +
Arnaud Launay is the server administrator. To get SSH access there, be a team member and find someone who already has access.
 +
 
 +
=== git mirrors (code) ===
 +
The [https://git.racktables.org/ git mirror server] exists to provide backups in case github.com goes kaput. The most up to date copies are available through both a web-interface (gitweb) and git clone URLs, for example:
 +
git clone https://git.racktables.org/racktables
 +
Cloning into 'racktables'...
 +
The server synchronizes each repository both hourly and after a new commit appears on github.com (via a webhook). It also keeps up to 90 days of backups for every repository.
 +
 
 +
This cannot be hosted with the other servers because the git mirror server notifies the MantisBT server so the source control integration fetches new changes into its database. However, the TCP connection does not establish within the same network, it needs to originate from outside.
 +
 
 
=== Statistics ===
 
=== Statistics ===
−
SF download stats work for the downloads. Google analytics works for the content of racktables.org and demo.racktables.org.
+
SF download stats work for the downloads. Google analytics was used in the past for racktables.org and demo.racktables.org, but was removed in 2019.
−
=== Everything else ===
+
 
−
SourceForge. SVN, Mantis, Wiki, racktables.org HTTP vhost, downloads and shell service. '''Backup of data hosted on SF needs to be pulled manually.'''
+
=== downloads ===
 +
SourceForge project FRS (File Release System, a CDN of sorts).
 
----
 
----
 +
 
==Backup procedure==
 
==Backup procedure==
−
===racktables-users===
+
===the backup directory===
 +
mkdir RackTables-project-backup-`date +%Y%m%d`
 +
cd RackTables-project-backup-`date +%Y%m%d`
 +
 
 +
===racktables-users mailing list===
 
Send an empty email with subject "who racktables-users" to ecartis@freelists.org and you will get a list of all subscribers. Save it to the backup directory as "racktables-users.txt".
 
Send an empty email with subject "who racktables-users" to ecartis@freelists.org and you will get a list of all subscribers. Save it to the backup directory as "racktables-users.txt".
−
===SourceForge data===
 
−
# Project page, "Project admin", "Features", "XML export": save the file into backup directory as "racktables_export.xml".
 
−
# Project page, "Project admin", "Features", "Hosted apps": save 4 files (mantisbt, mediawiki)*(Database,Files) into backup directory.
 
−
# Backup stuff accessible from SF SSH only:
 
−
<pre>ssh -t YOUR_SF_USERNAME,racktables@shell.sourceforge.net create
 
−
adminrepo --checkout svn
 
−
svnadmin dump /svnroot/racktables | gzip > racktables-svn-dump.bin.gz
 
−
adminrepo --discard svn
 
−
tar czf racktables.org.tar.gz /home/project-web/racktables/htdocs/
 
  
−
sf-help | fgrep 'ssh -p'
+
===Git repositories===
−
# Note the hostname and the port and copy the files from SF shell to your directory:
+
git clone --mirror git://github.com/RackTables/racktables.git racktables.git
−
scp -P 24006 YOUR_SF_USERNAME@shell4.sourceforge.net:~/racktables-svn-dump.bin.gz .
+
git clone --mirror git://github.com/RackTables/racktables-contribs.git racktables-contribs.git
−
scp -P 24006 YOUR_SF_USERNAME@shell4.sourceforge.net:~/racktables.org.tar.gz .
+
 
 +
===Bug tracker, wiki data, www===
 +
SSH into the host and create a new backup directory.
 +
mkdir ~/RackTables-apps-backup
 +
cd ~/RackTables-apps-backup
 +
 
 +
Copy the applications into the backup directory on the server. This is important because they contain uploaded files and configuration settings.
 +
tar czf bugs.tar.gz ~/www/bugs
 +
tar czf wiki.tar.gz ~/www/wiki
 +
tar czf www.tar.gz ~/www/www
 +
 
 +
Export the database for each application.
 +
mysqldump -u <username> -p <dbname> > bugs.sql
 +
mysqldump -u <username> -p <dbname> > wiki.sql
 +
 
 +
SCP the remote server directory to the local backup directory, then remove it from the server.
  
−
# in SF shell again
+
===Final check===
−
rm racktables-svn-dump.bin.gz racktables.org.tar.gz
+
Now the backup directory must have these files/directories:
−
shutdown
+
* racktables.git
−
</pre>
+
* racktables-contribs.git
−
Now the backup directory must have these files:
 
−
* racktables-svn-dump.bin.gz
 
 
* racktables-users.txt
 
* racktables-users.txt
−
* racktables.org.tar.gz
+
* RackTables-apps-backup
−
* racktables_export.xml
+
** wiki.tar.gz
−
* racktables_mantisbt_files.tar.bz2
+
** wiki.sql
−
* racktables_mantisbt_mysqldump.sql.bz2
+
** bugs.tar.gz
−
* racktables_mediawiki_files.tar.bz2
+
** bugs.sql
−
* racktables_mediawiki_mysqldump.sql.bz2
+
** www.tar.gz
−
Wrap this stuff into an arhive with a meaningful name and save it to a safe place:
+
'''SAVE THE BACKUP FOLDER TO A SAFE PLACE'''
−
<pre>
 
−
tar cf RackTables-project-backup-`date +%Y%m%d`.tar ./racktables*
 
−
</pre>
 

Latest revision as of 19:53, 25 February 2024

Mapping of assets

DNS

racktables.org domain is currently registered through GoDaddy, which also provides 2 free nameservers. Aaron manages the domain.

racktables.org mail

racktables.org mail is handled by zohomail.com. Zoho Mail offers free service for up to three e-mail addresses. Two mailing lists exist, "devteam" and "info", which are setup to forward all messages to every development team member.

racktables-users mail

FreeLists is a great free service without accompanying advertisements. The only drawback about it is that it is non-profit and can go down some day, hence make sure the list of subscribers is present in a recent backup tarball.

To manage the mailing list, go to this page and use the form to the left. Input your email subscribed to the list. If you don't remember your password, log in with an empty password, in this case the system will email you a one-time token to input on the next screen. Once logged in, you will see a very long SELECT with all lists of FreeLists. Click on the SELECT and press "r" to get to items starting with R. Select racktables-users and press "Select list" below. On the next screen press "Admin Menu". To manage individual subscribers, select one on the "Current List Users" SELECT and press "User Options". VACATION, MODPOST and NOPOST are the most frequently used flags, their meaning is explained inline.

Please think twice before changing any of the list-wide settings on the "Edit List Config" menu, these settings influence all subscribers of racktables-users.

Most web-sites (demo, bugs, wiki, www)

Arnaud Launay is the server administrator. To get SSH access there, be a team member and find someone who already has access.

git mirrors (code)

The git mirror server exists to provide backups in case github.com goes kaput. The most up to date copies are available through both a web-interface (gitweb) and git clone URLs, for example: 

git clone https://git.racktables.org/racktables
Cloning into 'racktables'...

The server synchronizes each repository both hourly and after a new commit appears on github.com (via a webhook). It also keeps up to 90 days of backups for every repository.

This cannot be hosted with the other servers because the git mirror server notifies the MantisBT server so the source control integration fetches new changes into its database. However, the TCP connection does not establish within the same network, it needs to originate from outside.

Statistics

SF download stats work for the downloads. Google analytics was used in the past for racktables.org and demo.racktables.org, but was removed in 2019.

downloads

SourceForge project FRS (File Release System, a CDN of sorts).

Backup procedure

the backup directory

mkdir RackTables-project-backup-`date +%Y%m%d`
cd RackTables-project-backup-`date +%Y%m%d`

racktables-users mailing list

Send an empty email with subject "who racktables-users" to ecartis@freelists.org and you will get a list of all subscribers. Save it to the backup directory as "racktables-users.txt".

Git repositories

git clone --mirror git://github.com/RackTables/racktables.git racktables.git
git clone --mirror git://github.com/RackTables/racktables-contribs.git racktables-contribs.git

Bug tracker, wiki data, www

SSH into the host and create a new backup directory. 

mkdir ~/RackTables-apps-backup
cd ~/RackTables-apps-backup

Copy the applications into the backup directory on the server. This is important because they contain uploaded files and configuration settings. 

tar czf bugs.tar.gz ~/www/bugs
tar czf wiki.tar.gz ~/www/wiki
tar czf www.tar.gz ~/www/www

Export the database for each application. 

mysqldump -u <username> -p <dbname> > bugs.sql
mysqldump -u <username> -p <dbname> > wiki.sql

SCP the remote server directory to the local backup directory, then remove it from the server.

Final check

Now the backup directory must have these files/directories:

  • racktables.git
  • racktables-contribs.git
  • racktables-users.txt
  • RackTables-apps-backup
    • wiki.tar.gz
    • wiki.sql
    • bugs.tar.gz
    • bugs.sql
    • www.tar.gz

SAVE THE BACKUP FOLDER TO A SAFE PLACE

Retrieved from "https://wiki.racktables.org/index.php?title=ProjectInfrastructure&oldid=957"