• Gitignore Tutorial by Atlassian
• Visual Git Cheatsheet
• Reference Git Documentation

Core commands:

• Create repository in folder git init
• Clone existing repository git clone URL
• Add any new files git add .
• Commit changes git commit -m 'Message'
• Push changes git push origin master or git push
• Force push changes (overwrites stuff) git push --force
• List all local branches git branch
• List all branches (incl. remote) git branch -a
• Create branch (but don't checkout) git branch NAME
• Create branch (and checkout) git checkout -b NAME
• Switch branch git checkout NAME
• Delete local branch git branch -d NAME
• Force delete local branch git branch -D NAME
• Delete remote branch git push origin --delete NAME
• Work on two branches at the same time (in VS Code):
    from master branch do this: git branch BRANCH-NAME and then git worktree add NAME-of-FOLDER BRANCH-NAME
    Git Worktree Add Example afterwards remove with git worktree remove BRANCH-NAME
• See log git log
• Nicely formatted log log --graph --date=short --pretty=format:"%h%x09%ad%x09%s"
• Purge file from commit history with BFG Repo-Cleaner
• Remove "ignored" files that were added before the gitignore was created git rm -rf --cached . and then git add . (make sure gitignore is uptodate)
• Delete "cached" files that were previously deleted from cache but are still in history git filter-branch --index-filter 'git rm -r --cached --ignore-unmatch "FILENAME"' -f HEAD (DANGEROUS - MAKE BACKUP)

January 2020

1. On local machine

• add ssh key to server's authorised keys ssh-copy-id -i ~/.ssh/id_rsa.pub user@server
• add ssh key to server's authorised keys ssh-copy-id -i c:\users\name\.ssh\id_rsa.pub user@server
• these might need a port too -p 1234

2. On server

• live site: /home/user/site
• bare repo: /home/user/site.git
• initialise bare repo: git init --bare
• alternatively use: git init --shared --separate-git-dir code.git html inside parent folder
• enter hooks folder: cd hooks
• create "post-receive" file: touch post-receive
• add the following script using nano post-receive:

  #!/bin/sh
  git --work-tree=/home/user/site --git-dir=/home/user/site.git checkout -f

• or use the post-receive script mentioned in this article: Simple automated GIT Deployment using GIT Hooks
• fix permissions of post-receive script: chmod +x post-receive

3. On local machine

• add new remote: git remote add server user@server:site.git
• add new remote (with port including via git config file setup): git remote add server ssh://user@server/full/path/to/site.git
• check remotes: git remote -v
• add the following lines to the local git config file (i.e. in .git/config as opposed to the global config in c:\users\name\.gitconfig):

  [alias]
    pushall = !git push origin master && git push server master

• make changes and push to both remotes: git pushall

Specify port:

• git remote set-url server ssh://user@server:port/absolute/path/to/site.git
• (if cmd complains about caching key and you can't click yes, run putty itself so you can click yes there)

Multiple users pushing to same repo on server:

• Create user account for "git": sudo adduser git
• su git
• cd
• mkdir .ssh && chmod 700 .ssh
• touch .ssh/authorized_keys && chmod 600 .ssh/authorized_keys
• Each user wanting to push to the server needs to add their public ssh key to the "git" user's authorized_keys file:
• From local machine: ssh-copy-id -i ~/.ssh/id_rsa.pub git@server
• From local machine: ssh-copy-id -i c:\users\name\.ssh\id_rsa.pub git@server

• see above
• then either have the code live in git home directory and all users push to that as git user (ssh://git@server/...)
• or the code lives in a user home directory and other users of the same git group can push to it (ssh://user@server/...)
• make sure folder permissions are correct (775/755 for folders/subfolders)

More info

• Simple automated GIT Deployment using GIT Hooks
• Git on the Server - Setting Up the Server
• How To Set Up Automatic Deployment with Git with a VPS

Add the following code to a pre-commit hook in the .git/hooks folder of your repo. This will generate html files from markdown content every time just before you commit.

Make sure you add styles inside a <style> block to a file called "styles". See pandoc command in script below.

#!/bin/sh

echo "Converting markdown files to html with Pandoc..."

for file in markdown/*.md 
do
  echo "Processing $file"
  name=$(basename ${file%%.*})
  # echo $name

  pandoc $file -f markdown -t html -o html/$name.html -s -H styles --quiet
  # --quiet = suppress warning messages
  # -s = standalone (full htmlm not snippet)
  # -H = include file in header
done

git add .
echo "... and stage generated html files."
      

• Pandoc User's Guide
• githooks.com
• The automation of documentation

August 2019

Secure Sockets Layer

• Let's Encrypt
• Let's Encrypt community forum
• certbot
• certbot for Nginx on Debian 9 (Stretch)
• certbot-auto for Nginx on Debian 8 (Jessie)
• Test your SSL settings
• Crontab help
• Certificates for localhost
  openssl req -x509 -out localhost.crt -keyout localhost.key \ -newkey rsa:2048 -nodes -sha256 \ -subj '/CN=localhost' -extensions EXT -config <( \ printf "[dn]\nCN=localhost\n[req]\ndistinguished_name = dn\n[EXT]\nsubjectAltName=DNS:localhost\nkeyUsage=digitalSignature\nextendedKeyUsage=serverAuth")

Nginx + Debian Stretch 9

• certbot for Nginx on Debian 9 (Stretch)
• install certbot: sudo apt-get install certbot python-certbot-nginx -t stretch-backports
• create certificate: sudo certbot --nginx
• certificates are found in: /etc/letsencrypt/live
• check certificates: certbot certificates
• renew certificates test:certbot renew --dry-run
• force renew certificates certbot renew --force-renewal
• expand/update certificates certbot certonly --cert-name example.com -d example.org,www.example.org
• revoke/delete certificates certbot revoke --cert-path /etc/letsencrypt/live/CERTNAME/cert.pem
• check crontab: crontab -l
• edit crontab: crontab -e
• restart cron: sudo /etc/init.d/cron restart
• certbot logs: /var/log/letsencrypt/letsencrypt.log
• cron job for auto-renew: 0 0,12 * * * python -c 'import random; import time; time.sleep(random.random() * 3600)' && certbot renew (unconfirmed)

Nginx + Debian Jessie 8

• Forum autorenew question
• certificates are found in: /etc/letsencrypt/live
• wget https://dl.eff.org/certbot-auto
• chmod a+x certbot-auto
• check certificates: /etc/certbot-auto certificates
• might need to add flag:--no-bootstrap
• renew certificates test:sudo /etc/certbot-auto renew --dry-run
• cron job for auto-renew: 0 0,12 * * * python -c 'import random; import time; time.sleep(random.random() * 3600)' && /etc/certbot-auto renew --pre-hook "service nginx stop" --post-hook "service nginx start"

fix ssl labs server rating ubuntu nginx link

August 2019

Daemons

• pm2
• pm2 GitHub source
• pm2 Ecosystem File
• Running Your Python Script with PM2
• Install pm2: npm i -g pm2
• Start a process via custom config: pm2 start app.json

  {
    "name": "name-of-app",
    "script": "name-of-script.js",
    "cwd": "./",
    "watch": true,
    "ignore_watch": ["node_modules", "logs", ".git", "data"],
    "error_file": "./logs/pm2errors.log",
    "out_file": "./logs/pm2out.log",
    "log_date_format": "YYYY-MM-DD HH:mm Z"
  }

• Start a process: pm2 start app.js
• List all processes: pm2 list
• Setup startup script: pm2 startup
• Stop a process: pm2 stop NUM/NAME
• Restart a process: pm2 restart NUM/NAME
• Delete a process: pm2 delete NUM/NAME
• Forever
• Example conf file for Forever
• Running Node Apps with Forever
• See also: forever-service
• Install Forever: npm i -g forever
• List all processes: forever list
• List all configs: forever config
• Start a process: forever start app.js
• Start a process with logs: forever start -l foreverLog.log -o foreverOutput.log -e foreverErrors.log app.js
• Stop a process: forever stop app.js
• Restart a process: forever restart app.js
• Start a process via forever.json: forever start forever.json

  {
      "uid": "name-of-node-app",
      "append": true,
      "watch": true,
      "watchIgnore": ["/absolute/path/logs",
                      "/absolute/path/irrelevant/stuff"],
      "colors": true,
      "script": "server.js",
      "sourceDir": "/absolute/path",
      "logFile": "/absolute/path/logs/forever.log",
      "outFile": "/absolute/path/logs/foreverout.log",
      "errFile": "/absolute/path/logs/forevererror.log"
  }

  or

  {
      "uid": "name-of-python-app",
      "append": true,
      "watch": false,
      "colors": true,
      "script": "wsgi.py",
      "command": "venv/bin/python",
      "sourceDir": "/absolute/path",
      "logFile": "/absolute/path/logs/forever.log",
      "outFile": "/absolute/path/logs/foreverout.log",
      "errFile": "/absolute/path/logs/forevererror.log"
  }

• Add to git post-receive hook: forever restart uid-of-process
• Might need to make sure you run these on the same user.

June 2022

General

• Securing a VPS
• Changing a VPS hostname
• Mozilla server sec checker: observatory.mozilla
• Cross origin resource sharing (CORS): enable-cors.org
• get Debian version lsb_release -d
• get Kernel version cat /proc/version
• When upgrading remember to re-install nginx, git, certbot, couch, python 3.8, etc
• Check free disk space in human readable form df -h
• Check what is using space in human readable form du -h
• As above but as summary du -h -s *
• Show running processes: ps -ef

Users

• Chmod Calculator
• Get all users getent passwd
• Get normal users getent passwd {1000..60000}
• Change file permissions chmod XXX file
• sudo adduser name
• Change user permissions chown user file
• Change permissions recurively chown -R user:group folder
• Change group permissions chgrp group file
• Change group permissions for all chgrp -R group folder
• Edit access rights nano /etc/ssh/sshd_config
• Restart ssh client after change /etc/init.d/ssh restart
• Add user to sudo group usermod -aG sudo username

NGINX

• /etc/nginx/sites-available
• /etc/nginx/sites-enabled
• /etc/nginx/snippets
• stop nginx: service nginx stop
• restart nginx: service nginx restart
• get nginx status (incl pid): service nginx status
• test settings: nginx -t
• create symlink from sites-available to sites-enabled: ln -s /etc/nginx/sites-available/name /etc/nginx/sites-enabled/name
• Nginx not restarting
• check running processes of nginx: ps aux | grep [n]ginx
• kill process: kill -9 1234 for each process!
• then start nginx again: service nginx start

Screen

• show all running screens: screen -ls
• start new screen with name: screen -S name
• attach to a running session: screen -x
• attach to session name: screen -r name
• detach from a session: Ctrl-a d
• kill a screen: screen -S NUMBER -X quit

Netstat

show ports used: netstat -plutn

Command-line Interfaces

June 2022

General

• Curl Manual
• Open cmd as admin 'Right-click' cmd executable
• My own Custom bash colours GitHub gist
    inclusive: .bashrc, .nanorc, git-prompt.sh, and settings.json
• Overview of nano's shortcuts
• Edit files with nano
• Appending contents of one file to another cat /absolute/path/to/file1 >> ./file2 (file2 being at current pwd)

Customisation

• Bash customisation tool
• Profile settings in Windows Terminal
• Create Vertical or Horizontal Panes in Windows Terminal using Alt Shift + or Alt Shift -
• You can now also change colour and names of tabs in Windows Terminal by right clicking the context menu!
• Close a pane using Ctrl Shift W.
• Customisation config file for new Windows Terminal in Customisation stuff in C:\Users\Fania\AppData\Local\Packages\Microsoft.WindowsTerminal_8wekyb3d8bbwe\LocalState\profiles.json
• Adding Git-Bash to the new Windows Terminal
• 

Open at location

• 'Shift-right-click' folder to open CMD there
• Use the 'Open-In-Terminal' script

Finding things

• Show all locations of file type -a NAME
• Show location of file or folder which NAME
• Show location of file or folder (if virtualenvwrapper-win is installed) whereis <NAME>

Reloading

• Reload terminal source ~/.bash_profile
• Reload terminal source ~/.bashrc
• Reload cmd SET PATH=%PATH%;C:\CmdShortcuts (unconfirmed)

Running Scripts

• Make sure Execution Policy is set appriately Set-ExecutionPolicy RemoteSigned
• Check Execution Policy with Get-ExecutionPolicy
• See How to allow scripts to run

• SVG Editor
• online editor
• Easings
• Cubic Beziers
• SVG JS library snap.svg
• SVG Background Patterns svgbackgrounds.com
• Broswer fallback svgeezy.js
• SVG Loading Animation
• SVG animations guide
• codepen

• MDN CSS Code Examples used in MDN articles and demos
• A truly monstrous async web chat using no JS whatsoever on the frontend CSS-Only Chat
• Create Diagonal Layouts Like It's 2020
• CSS Gradient Animator
• Animated Hover Gradient Demo
• mix-blend-mode: difference for automatic contrast colour for background

FlexBox

• darek kay cheatsheet
• malven cheatsheet
• TEST CSS FLEXBOX RULES (found this to be slightly buggy sometimes)
• Build awesome user interfaces with CSS flexbox Flexbox Patterns by CJ Cenizal

Grid

• Game to learn CSS Grids CSS Grid Garden
• grid by example
• flexible grids
• malven cheatsheet
• Jen Simmons Mondrian Grid Demo

Tools

• Delete unused CSS rules Purge CSS

SASS

• Reference SASS
• How to setup Sublime
• Edit scss file and then Command+B to compile
• Edit init.js file to use sass or css files; or if scripts are disabled edit individual import statements in html files

• debug several variables in one console log console.log({x,y,z});
• equivalent to doing console.dir(x); but for each x, y, and z
• Professor Frisby's Mostly Adequate Guide to Function Programming with JavaScript
• You Don't Know JS - A book series on JavaScript YDKJS by Kyle Simpson
• Document fragment vs innerHTML vs looped appendChild jsperf.com
• Callbackhell

Node

• Official Node.js website
• npm init to create package.json
• npm i XYZ or npm install XYZ to install XYZ into local "node_modules" folder
• node app.js` or node app` to run app
• npm rm XYZ to remove XYZ
• npm uninstall XYZ to remove XYZ
• process.stdout.write("Downloading " + data.length + " bytes\r"); to print to console in same line as before...
• process.stdout.write("Downloading " + data.length + " bytes\033[0G"); to print to console in same line as before...

Cross references - see also Pandoc & Git Hooks.

• Dillinger: For creating, editing and exporting markdown files in html.

March 2020

General Guides / Checklists / Roadmaps

• General Web.dev Guide to Learn CSS!
• Code prettifier prettier.io
• A resource of stuff that's Free for developers
• Front End Checklist
• Web skills overview see source on GitHub
• Developer Roadmaps at roadmap.sh and dev roadmap
• Progressive Web App Demo: What PWA can do today
• Things you can do with a browser in 2020 Browser 2020
• Writing todos todo.txt
• Microsoft Frontend Bootcamp
• No Design Development: A collection of tools for developers who have little to no artistic talent.

Tools / Resources

• RegExR Regular Expression Tester
• Learn CSS Grid Learn CSS Grid layout helper
• Box Shadow Generator Helps generate box shadows (and other things apparently)
• CSS Gradient Generator Helps generate css gradients (and all sorts of other things!)
• handbrake.fr open source video editor/trandcoder. lightweight and easy enough to use. multi platform too.
• EZGIF.com Video cropper
• online-video-cutter.com Video cutter and cropper
• AllTheFreeStockCollection of free media resources
• Table Generator: For creating, editing tables in Latex, html, markdown, etc.

Service Workers

• serviceworkers

Intersection Observers

• see here

Jokes

• No Code: The best way to write secure and reliable applications. Write nothing; deploy nowhere.

Search Engine Optimasation (SEO)

• metatags.io
• A free guide to HTML5 <head> elements
• Google Search Console / Webmaster
• Google Mobile Friendly Test

Favicons

• Favicon Generator
• Fontawesome Favicon Generator
• Maskable.app Editor for Chrome/Android maskable icons

Icons

• SVG to URL for CSS converter
• Tablericons
• Fontello Icon Fonts Generator
• Icomoon SVG icons
• HeroIcons high-quality SVG icons for UI development
• Zondicons free premium SVG icons
• CSS GG Minimalistic icon library Designed by code. Customizable & Retina-Ready icons — entirely built in CSS.

Fonts

• Golden Ratio Typography Calculator for optimal font sizes, line heights and line widths.

Images

• Remove BG Remove image background online
• Image Optimisation
• Ascii-Generator (image or text to ascii convertor)

Responsive Web Design

• Responsive Logos

Printing

• Print CSS stylesheets

Social / Sharing / Structured Data

• Open Graph
• Open Graph Checker with previews and template code
• Google Structured Data Test
• Google Structured Markup Test
• JSON Linked Data
• JSON LD Playground
• RDFa Linked Data
• RDFa Playground
• RDFa Lite Documentation

Interesting Sites

• Seeing Theory A visual introduction to probablity and statistics.
• Robert a.k.a. Robin

October 2017

• Texcount fix, Perl, Texcount.Perl - Comment out section on win stuff at top
• Table Generator: For creating, editing tables in Latex, html, markdown, etc.
• Latex Equation Editor: For creating, editing Latex equations.

March 2020

CouchDB

• couch daemon sudo sv status | stop | start couchdb
• find couchdb: localhost:5984
• couchdb GUI: localhost:5984/_utils
• edit config: nano /opt/couchdb/etc/local.ini
• restart couch: /etc/init.d/couchdb restart
• couch backup files: /var/lib/couchdb
• CouchDB Docs (Official Stable)
• CouchDB Nano for Node
• CouchDB Best Practices
• Bulk updates might need an increase in request body size due to error 413. add a line in nginx config file for couch domain: client_max_body_size 100M;
• files to backup are in /usr/local/var/lib/couchdb
• CouchDB Debian Buster
• CouchDB Nginx conf
• JSONBox.io free HTTP based JSON storage. See docs.

MariaDB

• How to connect: sudo mysql -u USERNAME -p (enter sudo pw first, and then the mysql user pw)

October 2017

• Vibrancy Extension to add opacity effect. Careful, this is partly broken atm (causes drag lag)
• Using SSH Keys in Visual Studio Code on Windows

Extensions

• Manage published extensions
• Extension Tutorial Hello World
• Publishing extensions with vsce
• vsce login user-name
• vsce package
• vsce publish minor/major
• TextMate Theme Generator

October 2017

• Haskell.org
• GHCi Lambda Prompt Tutorial
• Find "ghci.conf" file and enter: :set prompt "\x03BB> "
• Haskell Syntax
• Fixity

October 2017

WordNet

• Reference WordNet Interface for NLTK
• Import wordnet from nltk.corpus import wordnet as wn
• Loop through lemmas for l in synset.lemmas():
• Lemma name as string str(lemma.name())
• Antonyms synset.lemmas()[0].antonyms()
• Install NLTK 3.0.0b1 pip install -Iv https://pypi.python.org/packages/source/n/nltk/nltk-3.0.0b1.tar.gz#md5=f3f67fcda9bc76f57d91811a572c36e4

Data

• Reference NLTK Data
• Install nltk data: (1) python
• Install nltk data: (2) import nltk
• Install nltk data: (3) nltk.download()
• or nltk.download('stopwords') and nltk.download('wordnet')
• add path of nltk_data folder to system path
• or better yet add it to the local repo and code:
• import nltk
• nltk.data.path.append("PATH/TO/nltk_data")

April 2020

See also info on screen

Install Python 3.8

• How to Install Python 3.8 on Ubuntu, Debian and LinuxMint
• Check current version python -V
• Check current version pip -V
• Check current version python3.8 -V
• Create venv python3.8 -m venv venv
• Activate venv source venv/bin/activate

Random Useful Links

• How To Process Incoming Request Data in Flask
• Ascii Art library

Python WSGI ('Whiskey') Server

• Waitress documentation
• Set up a "wsgi.py" file:

  
  from app import app
  from waitress import serve
  if __name__ == "__main__":
      serve(app, host='127.0.0.1', port=8001)
  
  

• Then run server using python wsgi.py
• A Waitress & A Flask Running Flask Web Framework from within a Waitress WSGI Server
• Exposing your data science project to the world — Flask with Waitress!

Switch the /usr/bin/python link to point to current python link

• cd /usr/bin
• rm -f python
• ln -s /Library/Frameworks/Python.framework/Versions/Current/bin/python python

Pip

• Reference Pip
• Upgrade pip pip install -U pip
• Upgrade pip python -m pip install -U pip
• Install NAME pip install NAME
• Install NAME version 1.0.4 pip install NAME==1.0.4
• Install specific version pip install -Iv Link-to-tar.gz
• Uninstall NAME pip uninstall NAME
• Show installed packages pip list
• Show out-of-date packages pip list --outdated
• Upgrade NAME pip install --upgrade NAME
• To freeze local requirements pip freeze --local > requirements.txt
• To freeze requirements pip freeze > requirements.txt
• To install from requirements pip install -r requirements.txt

Virtualenv

• Python 3 reference Virtualenv
• Old reference Virtualenv
• Python 3: create a new venv python -m venv NAME
• Old: Create a new venv virtualenv NAME
• Activate NAME . NAME/bin/activate
• Activate NAME NAME\scripts\activate
• Deactivate deactivate

Virtualenvwrapper

• Reference Virtualenvwrapper-win
• Reference Virtualenvwrapper
• Make a new venv mkvirtualenv ENVNAME
• List all virtualenvs workon or lsvirtualenv
• Activate a virtualenv workon ENVNAME
• Deactivate virtualenv deactivate
• Remove virtualenv deactivate and then rmvirtualenv ENVNAME
• Bind ENVNAME to project setprojectdir <project_path>
• Bind ENVNAME to project setvirtualenvproject [virtualenv_path project_path]

Virtualenvwrapper-win not finding virtualenv-script.py:

• Edit C:\PythonXY\scripts\mkvirtualenv.bat with the following changes on lines 47, 48
• Line 47 virtualenv.exe %*
• Line 48 REM python.exe "%PYHOME%\Scripts\virtualenv-script.py" %ARGS%

Paths

• C:\Python27;C:\Python27\Scripts;
• Set folder for virtualenvs export WORKON_HOME=$HOME/.virtualenvs
• Set folder for virtualenv projects export PROJECT_HOME=$HOME/Desktop/PhD_Offline/Git_code
• source /Library/Frameworks/Python.framework/Versions/2.7/bin/virtualenvwrapper.sh
• PATH="/Library/Frameworks/Python.framework/Versions/2.7/bin:${PATH}"
• PYTHONPATH="/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/:${PYTHONPATH}"
• export PATH

October 2017

• Get battery report: powercg /batteryreport

• Dashboard Login
• Install the toolbelt
• Python Flask Guide
• Login heroku login (needs authenticator code)
• Git Push git push heroku master only in git bash !
• make sure reqs.txt is called requirements.txt

October 2017

• Forum Help
• Uncheck preference located under Edit > Preferences > Documents "Open Tool pane for each document"