***************** Deploying Ideatree to Ubuntu 16.04 with Django/Apache2/Postgresql ************************ For apache and django configuration use only: https://docs.djangoproject.com/en/2.0/howto/deployment/wsgi/modwsgi/ *including* the links to wsgi documentation. Other sites give incomplete information. For postgres, http://suite.opengeo.org/docs/latest/dataadmin/pgGettingStarted/firstconnect.html ****************** IMPORTANT TIPS: ******************************* - This installation assumes a deploy file called 'DEPLOY_INSTRUCTIONS' and a home directory of 'mysite'. CAUTION! WARNING! IMPORTANT: IF AT ANY POINT PUTTING http:// in a browser lists the files of a directory, don't leave it in that state!! Shut down apache: 'service apache2 stop'. - Leave the Chrome debugger open and with the 'Network' tab showing. After entering a URL, if you don't get the results you expected, click the request showing in the Network tab. If it has a note next to it saying 'from disk cache', open Chrome settings and clear the files cache (under 'Advanced' settings). IF you have trouble with postgres, for instance the 'postgres' default user not being created, follow the very important instructions on this page: http://johnmee.com/how-to-reinstall-postgresql-on-ubuntu repeated here: https://stackoverflow.com/questions/2748607/how-to-thoroughly-purge-and-reinstall-postgresql-on-ubuntu ******* End Tips ********************************* 1) Go to the home directory of the root user. Edit '.bashrc' and insert at the top: 'alias python=python3'. Run 'source .bashrc'. You might want to uninstall python 2.7 if it was automatically provided by the host service and won't be used for anything else. 2) In /var/www/ do: 'clone /root/ideatree.git/ ideatree' 2b) OR upload and unpack the tar file (mysite.tar.gz) to a directory beside the document root of the webserver. For example, if Apache2 has DocumentRoot set to /var/www/html, unpack mysite.tar.gz to /var/www/ideatree --- UPDATE --- with Ubuntu 18.04+, graphviz 2.40+ should be available, so just do 'apt install graphviz'. That's it! Steps 3, 4 not needed anymore! 3) Check if gcc is installed with: 'gcc -V'. IF NOT: run this command from https://gist.github.com/application2000/73fd6f4bf1be6600a2cf9f56315a2d91 : sudo apt-get update && \ sudo apt-get install build-essential software-properties-common -y && \ sudo add-apt-repository ppa:ubuntu-toolchain-r/test -y && \ sudo apt-get update && \ sudo apt-get install gcc-snapshot -y && \ sudo apt-get update && \ sudo apt-get install gcc-6 g++-6 -y && \ sudo update-alternatives --install /usr/bin/gcc gcc /usr/bin/gcc-6 60 --slave /usr/bin/g++ g++ /usr/bin/g++-6 && \ sudo apt-get install gcc-4.8 g++-4.8 -y && \ sudo update-alternatives --install /usr/bin/gcc gcc /usr/bin/gcc-4.8 60 --slave /usr/bin/g++ g++ /usr/bin/g++-4.8; Then select which alternate to use: sudo update-alternatives --config gcc And verify everything worked with: gcc -v 4) Change to the graphviz directory, and: ./configure make make install Verify it worked with: 'dot -V' 5) Install apache2: 'apt install apache2; apt install sites-blacklist' 6) Find django.conf in the 'mysite/installpieces' directory. Copy it to /etc/apache2/sites-available Enter the public ip into django.conf as the ServerName. Then, change the wsgi alias to 'testapp.wsgi' by un-commenting that lines in django.conf while commenting out the one that points to 'mysite/wsgi.py'. Edit the SiteName in django.conf to the public ip of the host. NOTE: When everything is running, remember to come back and set it to 'ideatreelive.com'. Check the paths of the VirtualHost directives that they match your installation. 7) Edit /etc/apache2/sites-enabled/000-default.conf and set the ServerAdmin. 8) Replace apache2.conf with the one provided in /var/www/ideatree/mysite/installpieces/apache2.conf. 9) Edit /etc/apache2/sites-enabled/000-default.conf again and set the DocumentRoot to '/var/ideatree/www/mysite'. 10) Run 'a2ensite django' to create a link from sites-available/django.conf to sites-enabled. 11) Check that pip is pip 3: 'pip --version'. 12) Install virtualenv using pip3. 12b) Create the environment: within /var/www/mysite/' run: 'virtualenv ideatreeEnv' 13) Start the virtual environment by running 'source ./activateVenv'. NOTE: before doing the next two steps, try installing wsgi with pip *while the virtualenv is activated*. IMPORTANT: Verify that wsgi is compiled for the same version of Python that the virtualenv is running. 14) - Install the apache apxs extention: 'apt install apache2-dev'. - Change to the wsgi source directory below the 'mysite/installpieces' directory. Run this command: './configure --with-apxs=/usr/bin/apxs --with-python=/var/www/ideatree/mysite/ideatreeEnv/bin/python3' - then: 'make' 'make install' 'make clean' 15) Take note of the path where mod_wsgi.so was installed. Use it in the following line and insert into /etc/apache2/apache2.conf, replacing '/usr/lib/...' with the actual path: ' LoadModule wsgi_module /usr/lib/apache2/modules/mod_wsgi.so ' 16) Start apache2 with 'service apache2 start'. 17) Verify that wsgi is working. In a browser, enter ' /myapp ' and see if 'Hello World' appears. If yes, go back to django.conf and comment out the Daemon Alias to 'testapp.wsgi' and uncomment the one for 'mysite/wsgi.py'. 18) While the virtual environment is running (You can tell by the presence of 'venv' on the left side of the command line prompt), change to the 'mysite/installpieces' directory and run 'pip install -r requirements.txt'. NOTE: Do NOT install the psycopg2-binary package. It will cause hard-to-track weird bugs. Just doing 'pip install psycopg2' requires that prerequisites be installed by 'apt' first. See https://www.psycopg.org/docs/install.html#prerequisites NOTE2: Psycopg 2.8 is not compatible with Django < 2.2 !! DB Migrations will fail. See https://github.com/divio/django-cms/issues/6666 19) Do additional apache2 and firewall configuration using directions for your web host. For example, for Digital Ocean: https://www.digitalocean.com/community/tutorials/how-to-install-linux-apache-mysql-php-lamp-stack-on-ubuntu-16-04 in that particular case, stopping after apache firewall configuration. No need to install MySQL or PHP. 20) Add the public ip to ALLOWED_HOSTS in ideatree/mysite/mysite/settings.py. 21) start apache: 'service apache2 start'. If there's an error, check syntax with 'apache2ctl configtest' 22) Go to http:///ideatree You should see the home page. NOTE: every change to python code will require an apache2 restart: 'service apache2 restart'. 23) Install Postgres using the instructions at: https://www.digitalocean.com/community/tutorials/how-to-install-and-use-postgresql-on-ubuntu-16-04 Make the new role that's created be 'ideatree', and say yes when asked if it's a superuser. Make the new database that's created be 'ideatree' (By convention, postgres expects a database and a superuser role with the same name.). 24) Go to /etc/postgresql/9.5/main and copy 'pg_hba.conf' to 'pg_hba.original'. Then edit /var/www/ideatree/mysite/pg_hba.conf and substitute the ip given with the public ip for this host. (But try a private ip if available.) Then copy /var/www/ideatree/mysite/pg_hba.conf to /etc/postgresql/9.5/main Configure postgres with the instructions at: http://suite.opengeo.org/docs/latest/dataadmin/pgGettingStarted/firstconnect.html BUT DON'T DO the pg_hba.conf part, that's already been taken care of above. 25) In mysite/settings.py, ensure the SITE_ID corresponds to the row in the database for the site (for allauth app). You'll have to query the database to make sure. 26) Go back to /etc/apache2/sites-available/django.conf and set SiteName to your domain, e.g. 'www.example.com', then 'service apache2 restart'. 27) Also in /etc/apache2/sites-available/django.conf at the very top, change *:80 to *:443. 28) Install HTTPS certificate security using instructions at: https://www.digitalocean.com/community/tutorials/how-to-secure-apache-with-let-s-encrypt-on-ubuntu-16-04 29) Put the domains that will point to this installation, e.g. 'example.com' and 'www.example.com' into 'ALLOWED_HOSTS' in the file 'var/www/mysite/mysite/settings.py'. 30) From the /var/www/mysite directory, (check that the virtual environment is still in effect) run: 'python manage.py migrate sites' 'python manage.py migrate' 31) IMPORTANT: In /var/www/mysite/mysite/settings.py, change Debug to False, then 'service apache2 restart'. 32) Change file permissions as described in As a minimum: https://serverfault.com/questions/357108/what-permissions-should-my-website-files-folders-have-on-a-linux-webserver For more security: https://stackoverflow.com/questions/16408079/secure-django-file-permissions In short: from the 'var/www' directory, do: chown -R ron ideatree/ (or wsgiuser ideatree/) chgrp -R www-data ideatree/ chmod -R 750 ideatree/ chmod g+s ideatree/ But two places need write-ability by Apache: a) Go to /var/www/ideatree and do: 'chmod 770 ideatree.log' b) Go to /var/www/ideatree/ideatree/templates/ideatree and do 'chmod g+w output' (make the output directory first with the permissions above, if necessary) 33) Go to https:// You should see the home page, IF the DNS records have been updated at the domain registrar (or on the host server if that's where the DNS records are located) and if there has been enough time (30 minutes to a few hours) for the records to be propagated worldwide. Use dnschecker.org to check propagation. Use 'dig www.ideatreelive.com' to check whether DNS is working. 34) Go to https:///ideatree/admin You should be able to sign in with username 'admin' and the same password used when creating the database, above. 35) If you started with an empty database rather than copying the database from some other installation, you need to initialize the permissions database table. It tells what people can do with various objects in a map. - uncomment the line in ideatree/urls.py for 'initPerms/'. - run 'service apache2 restart' - Log in to ideatree (not to ideatree/admin) as 'admin', with the password you set for the superuser you created when installing the database above. - Manually replace everything in the URL after 'ideatree/' with 'initPerms', as in: 'https:///ideatree/initPerms' - Press enter. YOu should get a blank white screen. - Click the back button to go back to Ideatree. - DON'T FORGET to comment out the line again in ideatree/urls.py - run 'service apache2 restart' 36) Review security measures at: https://www.digitalocean.com/community/tutorials/an-introduction-to-securing-your-linux-vps 37) HIGHLY IMPORTANT: It is possible under some conditions to run up huge bills on Firebase without knowing it. Be sure the cron job that cleans out stale Firebase data every hour is running. In addition, there are edge conditions in Firebase that may affect you. See https://pamartinezandres.com/lessons-learnt-the-hard-way-using-firebase-realtime-database-c609b52b9afb And https://stackoverflow.com/questions/41478849/firebase-keepsyncedtrue/42034913 And https://stackoverflow.com/questions/40190234/firebase-what-is-the-difference-between-setpersistenceenabled-and-keepsynced If you choose to make these configuration changes to Firebase, they would go in function step2() of authClient2Firebase.js *********** Troubleshooting ************************ 1) If you continually get 'ip address of server not found', run 'dig ' and 'ping '. If these work, change the properties of the wireless adapter on your computer to use specific DNS servers rather than the local ISP DNS server, which isn't always up to date with DNS changes you've made at the domain registrar. Look up how to change wireless properties on your type of computer, then change the DNS server to 8.8.8.8 (Google) or 208.67.220.220. 2) For a list of really useful networking commands, see: https://unix.stackexchange.com/questions/50098/linux-network-troubleshooting-and-debugging