Mac Installation Instruction

This gist contains all the information I need when I reset my MacOS. There are some overlaps between this gist and my super cheatsheet, but the super cheatsheet is more about specific commands and usage.

Some good references are: macOS setup guide, Setting up a Brand New Mac for Development, and dev-setup.

Installation Guide

Backup and Reinstall

Commit and push all git repos
Download express VPN client
Backup old OS
Copy local folders and documents
- .emacs and .emacs.d
- ~/desktop
- ~/documents
- Others
Erase current system and reinstall OS
Copy things from hard drive to new OS
Install express VPN on new OS
Open gists and follow the installation procedures

System Preference

Turn on dock magnification and hide
Setup dock folders to list view
Setup clock to analogy view
Setup keyboard F1, F2 key etc. as it is: Preferences -> Keyboard -> Select Use F1, F2 etc. keys as standard function keys...
Setup trackpad tap to click
Setup trackpad tap to drag: Preferences -> Accessibility -> Mouse * Trackpad -> trackpad Options… -> Enable dragging
Setup spotlight shortcut: Preferences -> Spotlight -> Keyboard shortcut -> deselect Show Finder search window -> Click Show Spotlight search shortcut to change -> Option + Command + Space
Setup language change: Preferences -> Keyboard -> Shortcuts -> Input Sources -> Click shortcut key binding to change -> Command + Space
Enable apps from any source: Open Terminal -> $ sudo spctl --master-disable
Turn off rearranging Spaces: Preferences -> Mission Control -> Turn off Automatically rearrange Spaces...
Disable dashboard as separate space: Preference -> Mission Control -> Select Dashboard: As Overlay

XCode

Install XCode developer tool using $ xcode-select --install
Download and install XQuartz

Terminal

Setup terminal theme to Grass, font size 22
Setup .bash_profile
- Comment out lines with java 10 and virtualenv to make it work at first

Homebrew

Install Homebrew, follow here
- Install cask via $ brew tap caskroom/cask
- Use $ brew search <package> to search available apps
Brew install mas, to enable App store intallation via terminal
- $ brew install mas
- $ mas signin [email protected] if not sign in
- $ brew bundle to install all apps specified in Brewfile
- See this webiste and this website for reference

Sublime

Install Sublime via $ brew cask install sublime-text
Add subl to bash using $ ln -s "/Applications/Sublime Text.app/Contents/SharedSupport/bin/subl" /usr/local/bin/subl, test subl by subl --help
Install Package Control
ctrl+shift+p to open package control console
In the console, type install package and hit enter to open package installer
Install predawn theme
Install following packages:
- Markdown Preview for markdown preview
- Emacs Pro Essential for Emacs key bindings
- Anaconda for python editing environment
  - Open Preferences -> Package Settings -> Anaconda -> Settings - Default
  - Update "python_interpreter": "python" as "python_interpreter": "C:/Python27/python.exe",,
- GitGutter for mark git changes
  - Specify git.exe location in GitGutter settings Preferences > Package Settings > GitGutter > Settings - User
- SublimeREPL for using R
  - Install R and add R to the %PATH% environment variable
  - Go to Preferences > Package Settings > SublimeREPL > Settings-User, add the R executable to the PATH
```
{
    "default_extend_env": {"PATH": "{PATH};C:\\Program Files\\R\\R3.2.3\\bin\\x64"},
    "show_transferred_text": true
}
```
  - Open up R console by pressing Ctrl+Shift+P and typing REPL R
  - Run the current line or your selected lines, press Ctrl+Shift+,,l
  - Run the selected text, press Ctrl+Shift+,,s
  - Run the entire file, press Ctrl+Shift+,,f
  - If SublimeREPL Eval doesn't work, following the answer at this website
- SideBarEnhancements for side bar functionality
- Markdown Editing for markdown syntax highlight and more
- ConvertToUTF8 for Chinese character display, this package requires to install another plugin Codecs33

Install packages offline

Download package .zip files from github
- One can find url of packages via package control of an onlined Sublime
Unzip the package files
Rename the zip files as xxx.sublime-package
Put some of the renamed zip files into Sublime Installed Package folder and some of the unzipped folders into Sublime Packages folder
- Installed Packages
  - Predawn
  - MarkdownEditing
  - ConvertToUTF8
  - sublemacspro
  - SideBarEnhancements
- Packages
  - Package Control
  - Predawn
  - SideBarEnhancements

Git

To use brewed Git instead of Apple pre-installed Git, add its path to ~/.bash_profile
Create git config file using $ touch ~/.gitconfig and then edit it $ edit ~/.gitconfig

[user]
    name = First Last
    email = [email protected]
[github]
    user = username
[alias]
    a = add
    ca = commit -a
    cam = commit -am
    s = status
    pom = push origin master
    pog = push origin gh-pages
    puom = pull origin master
    puog = pull origin gh-pages
    cob = checkout -b
[credential]
    helper = osxkeychain

See this website for reference
To enable git large file system, do $ git lfs install (only once)
Clone Git projects

Mac Default Apps

Mail
Books
iTunes
Photos

Cask Installed Apps

Dashlane, hotmail
OneDrive, hotmail
Wunderlist, google
Slack, columbia
Pocket, google
Trello, hotmail
Spark
Things
东方财富, 923
YoudaoDict, 163
Dropbox, hotmail
Airtable, google
Cyberduck, columbia

Other Apps

Office 365, hotmail
EndNote X8
Little Snitch (to block EndNote connections)
NeteastMusic, 163
duet
MindNode
Storyist
Printer driver
幕布, 923

Work Related

Opera, hotmail (work browser)
Wind, institution ID
幕布
Dia
Pulse Secure (FuDan U VPN)

Safari Extensions

Dashlane
OneNote web Clipper
Pocket

CPlusPlus

C++ comes with XCode
Verify C++ using $ c++ --version in bash
Clion is brewed
- Set keymap to emacs
- Set font size to 18
- Set appearance default fonts size to 14

Java

Java is casked
Verify Java using $ java -version in bash
IntelliJ Idea is brewed
- Set keymap to emacs
- Set font size to 18
- Set appearance default fonts size to 14

Python2

Python2 is brewed
Python2 is deprecated, will be terminated on Jan. 1, 2020

pip

Update pip and setuptools using $ pip install --upgrade setuptools and $ pip install --upgrade pip

pip install -r requirements.txt to install system-level packages specified in requirements.txt

ipython==4.2.1
jupyter==1.0.0
numpy==1.14.2
pandas==0.23.0
pipdeptree==0.12.1
matplotlib==2.2.2
scipy==1.0.0
virtualenv==16.0.0
virtualenvwrapper==4.8.2
jupyterthemes==0.20.0

Virtual Environment

Install virtualenv, virtualenvwrapper, and virtualenvwrapper.el for Emacs
virtualenv and virtualenvwrapper are installed via pip requirements

Enable virtualenvwrapper by adding or uncommenting following lines in ~/.bash_profile

export WORKON_HOME=$HOME/.virtualenvs
export PROJECT_HOME=$HOME/Devel
source /usr/local/bin/virtualenvwrapper.sh

Reload terminal via source ~/.bash_profile
$ cd to the project directory, use mkvirtualenv nameofenv to initialize a virtual environment
Use setvirtualenvproject to link the project with the virtual environment

Install packages for virtual environment qt via pip install -r requirements-qt.txt

ipykernel==4.8.2
bokeh==0.12.15
lxml==4.2.1
mysql-connector-python==8.0.11
pandas-datareader==0.6.0
pipdeptree==0.12.1
seaborn==0.8.1
SQLAlchemy==1.2.7
statsmodels==0.9.0
bs4==0.0.1
tushare==1.1.7
pykalman==0.9.5
quandl==3.4.0

IPython

From IPython 4.x, IPython does not come with notebook feature, all web notebook features are integrated into Jupyter. So if we want to use IPython Notebook, we need to install Jupyter
Note that IPython 5.0 and later are not compatible with Emacs, and IPython 6.x requires Python3
One possible fix for IPython 5 in Emacs is shown here (tested, not working for my .emacs setup)
Another quick fix is to use pip install --force-reinstall ipython==4.2.1 to downgrade IPython. This is usually needed for virtual environment management.
- To use IPython interpretor in Emacs as usual, download IPython 4.2.1 via $ pip install ipython==4.2.1
- To use Jupyter Notebook, install via $ pip install jupyter, it does not conlict with IPython4.2.1.

Jupyter Notebook

Jupyter Notebook has been installed via pip requirements
If you want to use Jupyter Notebook under virtual environment:
- Inside virtual environment, install ipykernel using $ pip install ipykernel
- Create new kernel using $ ipython kernel install --user --name=projectname, where projectname is qt in my case
- When open a notebook within virtual environment, remember to switch kernel to the new kernel just created named projectname.
- See this website for reference.
- One observation is that when outside the virtual environment, the new kernel is still working. That is, the modules installed under virtual environment are still callable even outside the virtual environment using the new kernel. For example, without activating virtual environment qt-env, the qt kernel is still working for ~/Documents/Projects/quant-trading/notebooks/EP Chan Book.ipynb.
If you want to use jupyter_contrib_nbextensions, install the package via pip install jupyter_contrib_nbextensions, then jupyter contrib nbextension install --user, then jupyter nbextension enable codefolding/main where codefolding/main is the path to your source code folder.
To add theme to Jupyter Notebook, one can install pip install jupyterthemes, use jt -l to check theme list, and use jt -t solarizedl to change theme.
To use Jupyter Notebook as slides, in the notebook, change View -> Cell Toolbar to Slideshow.
- Assign slide type of each cell in the notebook.
- Render the slides using !jupyter nbconvert test_ppt.ipynb --to slides --post serve --SlidesExporter.reveal_theme=serif --SlidesExporter.reveal_scroll=True --SlidesExporter.reveal_transition=none within the notebook.
- Check this post for details and this website for nbconvert documentation.

Python3

Python3 is brewed
All unversioned command such as $ pip pointing to Python3

pip

Update pip3 and setuptools using $ pip3 install --upgrade setuptools and $ pip3 install --upgrade pip

pip3 install -r requirements.txt to install system-level packages specified in requirements.txt

ipython==4.2.1
jupyter==1.0.0
numpy==1.16.1
pandas==0.24.0
pipdeptree==0.12.1
matplotlib==3.0.2
scipy==1.2.0

Virtual Environment

read this post for some info
To use multiple versions of Python, pyenv is a solution; for pyenv and pyenv-mode, see this website and this website
Use multiple versions of python in Emacs, read this ref
A more populor solution for virtual environment is pipenv; see this post for a nice introduction

R

R is brewed
Emacs is linked with R and is the major editor for R
R dependency gcc uses a different C++ standard (libstdc++), not the usual MacOS C++ standard (libc++)
- This may cause runtime error
- See my gist coding problems re details of this warning
Install R packages in ~/Documents/RLib instead of default location to avoid packages lost after R upgrade.
Create a file ~/.Rprofile via $ touch ~/.Rprofile and specify .libPaths("~/Documents/RLib/") using $ edit ~/.Rprofile
This command will always append the external library ~/Documents/RLib/ to .libPaths() upon cession startup
Test using .libPaths() and require(ggplot2)
Frequent R packages

# in ~/Documents/RLib/
ggplot2         2.2.1 
dplyr           0.7.5
knitr           1.20
rmarkdown
caret
RcppRoll        
e1071
reshape         0.8.7
randomForest
SuperLearner
kernlab

Emacs

Install Emacs via downloaded binary file
Add .emacs file
Paste previous .emacs.d/plugins to ~/.emacs.d/plugins
Work properly for Emacs24, Emacs25.3.1 and Emacs26.1
Chao's python in emacs setup and Purcell's emacs setup

Setup exec-path-from-shell

Setup a set-exec-path-from-shell-PATH function to make Emacs $PATH the same as bash $PATH
Define an elisp function in .emacs

;; Use the value of $PATH constructed at shell's creation
;; (done only when emacs started with window-system)
;; mostly the r startup nil problem is caused by not having this done
;; see [this website](https://howtos.gattaz.net/2016/01/22/install-emacs-ess-and-org-mod-on-os-x/) for reference
(defun set-exec-path-from-shell-PATH ()
    (let ((path-from-shell (replace-regexp-in-string
                                "[ \t\n]*$"
                                ""
                (shell-command-to-string "$SHELL --login -i -c 'echo $PATH'"))))
                (setenv "PATH" path-from-shell)
                (setq eshell-path-env path-from-shell) ; for eshell users
                (setq exec-path (split-string path-from-shell path-separator))
        )
)
(when window-system (set-exec-path-from-shell-PATH))

Emacs R

M-x R will start R console in Emacs
install R packages in Emacs: install.packages("car", repos='http://cran.us.r-project.org') library(car)

Emacs R markdown

M-x package-install RET markdown-mode RET to install markdown mode
M-x package-install RET polymode RET to install polymode for R markdown
M-x package-install RET poly-R RET to install poly-R for R markdown
update init file to 03/16/2019 version
this is a good reference
to render, first install pandoc on Mac brew install pandoc
call rmarkdown pakcage library(rmarkdown), render file render("filename.Rmd")
markdown-mode can only read files with .markdown extension, not .mdown extension

Python mode

install IPython using pip $ pip install jupyter or $ pip install ipython
- IPython 5.0 and later do not work with Emacs
- In order to use IPython in Emacs, we need to download ver4.2.1 via $ pip install ipython==4.2.1
download recent python-mode.el, put in the .emacs.d folder
in terminal, type which ipython to get IPython path
in .emacs file, add a line (setq-default py-shell-name "ipython"), substituting "ipython" with IPython path

LaTeX mode

the computer should have LaTeX installed
install auctex in Emacs: M-x package-install RET auctex RET
.emacs file should have LaTeX/auctex setup
Type C-c C-c three time to compile .tex source file
Type C-c C-c again to view the output

LaTeX Preview Pane

install this package via MELPA M-x package-install latex-preview-pane
enable it in .emacs file by adding (latex-preview-pane-enable)
enable it per buffer by M-x latex-preview-pane-mode
update preview by M-p
if .pdf file cannot be generated, it is because DocView major mode lacks ghostscript. Install ghostscript using $ brew install ghostscript to fix
see this webpage for reference

Common Lisp mode

install Allegro CL on the computer
setup .emacs file, see this site for reference
activate common lisp mode using M-x run-lisp

cheat-sh.el

cheat-sh.el is an Emacs mode for cheat.sh
It is a super cheatsheet of all kinds of programming languages
Install via M-x package-install RET cheat-sh RET
Test it functionality using M-x ch-sh-list RET RET to check all sheets

Spell check

install aspell on Mac OS with homebrew brew install aspell or Windows
setup .emacs file, see this site
M-$ checks for single word, C-f6 enables flyspell mode

Git-gutter

install git-gutter via MELPA M-x package-install RET git-gutter
add a line to .emacs to enable git-gutter (global-git-gutter-mode +1)

py-autopep8

install via M-x package-install RET and py-autopep8 RET
Upon saving Python file, py-autopep8 will be executed on the current buffer to format the code

MySQL mode

To start MySQL in Emacs, type M-x my-sql-server1 to choose server or M-x sql-mysql to use default parameters
Default parameters are specified in .emacs

virtualenvwrapper.el

Install using M-x package-install virtualenvwrapper
Setup .emacs to use it

Issues with MacOS

MacOS 10.15 requires file access permission for Eamcs
See this post to find a solution

MATLAB

Download and install MATLAB from official website
Move default working directory to ~/Documents/Projects/MATLAB/
Add matlab-solarized theme to the working directory and use setupSolarized('dark') to activate
Change font size to 18
To remove the default folder generated by MATLAB, set userpath('/Users/yourname/Documents/Projects/MATLAB')

LaTeX

BasicTeX Setup

If using BasicTeX, install tlmgr to manage missing packages
If using MacTeX, most packages are preinstalled, package management can be done through TeX Live Utility
BasicTex is casked
Set path variable in .bash_profile
Use tlmgr to install packages
Install tlmgr in bash via $ sudo tlmgr update --self
Install packages via $ sudo tlmgr install packname
If packname is not found, you can always search it by $ tlmgr search --global --file packname
- For example, search optparams.sty, you do $ tlmgr search --global --file optparams.sty, it gives us
```
tlmgr: package repository http://mirrors.opencas.cn/ctan/systems/texlive/tlnet:
texmf-dist/tex/latex/sauerj/optparams.sty
```
- So the package name is sauerj
Build the .tex and .bib files in bash $ pdflatex main.tex, $ bibtex main, $ pdflatex main.tex, and finally $ open main.pdf (note, there are four commands).
See this post and this blog for references.

Update BasicTeX

BasicTeX will update every year and does not provide an upgrade script for the current version. That means, users need to update it manually (fresh install the new version). Check /usr/local/texlive/ to find out the current version BasicTeX.
The caveat of a fresh install is that all the tlmgr packages will be lost. To avoid this, you can manually upgrade texlive by following the instruction. However, it seems very complex and is not recommended by the texlive team.
The other option is to stay with the current version, use $ sudo tlmgr option repository ftp://tug.org/historic/systems/texlive/2017/tlnet-final
The normal tlmgr package repository is http://mirrors.rit.edu/CTAN/systems/texlive/tlnet

MySQL

MySQL is brewed
Add aliases to bash by edit ~/.bash_profile and modify path variable based on MySQL location
Restart computer, and start MySQL server by $ mysql.server start
Initially, MySQL does not have a password (for MySQL < 5.7), one can use it by $ mysql -u root
Then, change the root password using $ mysqladmin -u root password newpassword
In bash, type $ mysql -u root -p then type password to use MySQL
See this post for reference

Jekyll

Jekyll is a static site generator, which can be used to create and manage blog posts automatically. It requires Ruby, which comes with MacOS 10.13.
Poole is a nice simple Jekyll theme that makes setup even more easier.
To use them, we need to install Ruby, we don't want to use Mac default one
Ruby can be brewed
Then, modify ~/.bash_profile to add Ruby in path variable
- First, install Jekyll using $ gem install jekyll.
- Second, clone Poole repo to local and rename it as blog.
- Third, cd to blog directory, remove the cloned remote server using $ git remote remove origin.
- Fourth, create a new repo on GitHub named blog, and add its remote to the local blog directory via $ git add remote NEW_REMOTE_URL.
- Then, running it locally via $ jekyll serve and open http://localhost:4000 to test the functionality.

In order to integrate Jekyll blog to GitHub Page, one need to modify _config.yml file as following:

# Setup
title:               Zhizun Zhang's Blog
tagline:             
url:                 "https://zazhang.github.io"
paginate:            2
baseurl:             "https://zazhang.github.io/blog/" # comment out this line when testing locally at http://localhost:4000

# Conversion
markdown: kramdown
highlighter: rouge
lsi: false
excerpt_separator: "\n\n"
incremental: false

# Markdown processors
kramdown:
input: GFM
hard_wrap: false
auto_ids: true
footnote_nr: 1
entity_output: as_char
toc_levels: 1..6
smart_quotes: lsquo,rsquo,ldquo,rdquo
enable_coderay: false

# Gems
gems:
  - jekyll-paginate
  - jekyll-gist

Poole is developed on Jekyll 2. If Jekyll 3 is installed, you need to install Jekyll-paginate and Jekyll-gist separately and include them in the _config.yml in order to use Poole.
Refer this post for setting up Poole, Atom, RSS, Google Analytics, and comments section.
Refer this post for pure Jekyll setup and detailed Jekyll structure explanation.
Refer this post for Jekyll and theme setup.

Blog Editing

Above configuration uses kramdown for Jekyll markdown processor. The syntax can be found in this post and this post.
To use LaTeX-like math equations, you can include MathJax with Jekyll. See this post for an instruction.
- Basically, only adding the JavaScript snippet to the .markdown post works. All other solutions do not work.

To add an index of all blog posts, modify index.html under the Jekyll directory as following,

<ul>
  {% for post in site.posts %}
    <li>
      <a href="https://zazhang.github.io/blog{{ post.url }}">
        {{ post.title }}
      </a>
      + <time datetime="{{ post.date | date: "%Y-%m-%d" }}">{{ post.date | date_to_long_string }}</time>
    </li>
  {% endfor %}
</ul>

<div class="archive_link">
  <a href="/archive">View all previous posts in archive &raquo;</a>
</div>

<script type="text/javascript">
//<![CDATA[
(function() {
    var links = document.getElementsByTagName('a');
    var query = '?';
    for(var i = 0; i < links.length; i++) {
      if(links[i].href.indexOf('#disqus_thread') >= 0) {
        query += 'url' + i + '=' + encodeURIComponent(links[i].href) + '&';
      }
    }
    document.write('<script charset="utf-8" type="text/javascript" src="http://disqus.com/forums/newbambblog/get_num_replies.js' + query + '"></' + 'script>');
  })();
//]]>
</script>

Note, in order to use with GitHub Page, append your baseurl (in my case, https://zazhang.github.io/blog) in front of {{ post.url }}.
See this post and this gist for reference.

Appendix

Developer Tools

Sublime
C++/Clion
Java/IntelliJ Idea
Latex
Python
R
Emacs
Matlab
SQLite, comes with MacOS
MySQL
IPython
Jupyter notebook
ACLExpress

App Store

OneDrive
Dashlane
Wunderlist
Dr. Unarchiver
Spark
Slack
Magnet
Pocket
YoudaoDict
Dr. Cleaner
东方财富

Other Essential Tools

Dropbox
Office 365
EndNote
Little Snitch
Airtable
NeteastMusic
Diablo II
Cyberduck
duet
Grammarly
Beamer
百度网盘
MindNode
Storyist

zazhang/macOS-installation-instruction.markdown

Mac Installation Instruction

Installation Guide

Backup and Reinstall

System Preference

XCode

Terminal

Homebrew

Sublime

Install packages offline

Git

Mac Default Apps

Cask Installed Apps

Other Apps

Work Related

Safari Extensions

CPlusPlus

Java

Python2

pip

Virtual Environment

IPython

Jupyter Notebook

Python3

pip

Virtual Environment

R

Emacs

Setup exec-path-from-shell

Emacs R

Emacs R markdown

Python mode

LaTeX mode

LaTeX Preview Pane

Common Lisp mode

cheat-sh.el

Spell check

Git-gutter

py-autopep8

MySQL mode

virtualenvwrapper.el

Issues with MacOS

MATLAB

LaTeX

BasicTeX Setup

Update BasicTeX

MySQL

Jekyll

Blog Editing

Appendix

Developer Tools

App Store

Other Essential Tools