TSYSGroupBOD/TSYSGroupHandbook
2
0
Fork 0
Code Pull Requests Actions Packages Projects Releases Activity

merging in material that will be folded in

Browse Source
This commit is contained in:
Charles N Wyble - admin
2021-08-03 21:48:37 -05:00
parent ca1e9cebd4
commit e6f1eae94f
71 changed files with 5990 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

65
SourceMaterial/engineering/beginners-guide/developer-workflow.md Normal file
View File

@@ -0,0 +1,65 @@
---
title: 4. Developer Workflow
sidebar: Handbook
showTitle: true
hideAnchor: true
---
If you haven't already, it's worth your time to read [Contributing to PostHog](https://posthog.com/docs/contributing).
Most developers use either [vscode](https://code.visualstudio.com/) or [pycharm](https://www.jetbrains.com/pycharm/) but
you are free to use whatever IDE makes the most sense to you.
## Backend w/ Vscode
1. Create a git branch
2. Start PostHog with `bin/start`
3. Open app in Chrome and login
4. Open Chrome devtools to network tab
5. Navigate to scene (aka screen or page) and click on the area of interest
6. Find network request in devtools and find request
- Request maps to ./posthog/api/*.py, i.e. http://localhost:8000/api/insight/funnel/?insight=FUNNELS -> ./posthog/api/insight.py:197
7. Make code changes including tests
- Use [print()](https://realpython.com/python-print/) as needed for debugging
- Some developers prefer [Pycharm](https://www.jetbrains.com/pycharm/) for local development
8. Run backend tests
- `bin/tests posthog` runs only posthog tests excluding ee tests
- `./bin/tests ee/clickhouse/queries/test/test_trends.py -k test_active_user_math` for specific tests
9. Commit changes to git branch
10. Open PR for review
- Include Github issue number `#1234` which Github will automatically link for you
## Frontend w/ Vscode
1. Same as backend 1-5
2. Find frontend code, i.e. `frontend/src/scenes/insights/Insight.tsx`
3. Use `console.log` liberally
3. As of writing, there are no unit tests for the frontend although we do have integration tests for the frontend via Cypress
4. Same as backend 9-10
## Alternative: Pycharm
Some developers prefer to use [Pycharm](https://www.jetbrains.com/pycharm/) and for
good reason. While there are many benefits, below you'll find a few keys benefits.
1. `Debugging and no print() statements` this is probably the biggest win in my opinion.
Since we are learning a new codebase there is no shame in having an assistant. Pycharm
will show you the call stack and variable values. This is huge for understanding what
is going on.
2. `Code navigation` when you are new to a codebase, moving easily through the code
can be a real challenge, especially when there are multiple layers of abstraction.
Pycharm allows you to Ctrl+Click nearly all methods to jump to their definitions.
While editors like vscode have a similar feature, you'll find that Pycharm works
in cases where vscode does not.
3. `Run configurations` make starting celery, django, and webpack services simple. It's
mostly just clicking things.
4. `Excellent TypeScript support` with code completion and type checking directly in your
editor.
5. `Click instead of type` which means that you spend much more time typing code than
running commands. Nearly everything in Pycharm is clickable.
Pycharm offers a try it for free 30-day trial. It's recommended that you use it for at least
that amount of time before you buy. I recommend watching [The Future of Programming](https://www.youtube.com/watch?v=8pTEmbeENF4)
that will blow your mind and perhaps give you a new perspective on tools like these.
**[Next: Technologies to learn](technologies-to-learn)**

27
SourceMaterial/engineering/beginners-guide/getting-started.md Normal file
View File

@@ -0,0 +1,27 @@
---
title: 3. Getting Started
sidebar: Handbook
showTitle: true
hideAnchor: true
---
## First goals
1. Set up your dev environment and configure with your IDE
2. Get PostHog running locally on Postgres: [http://localhost:8000](http://localhost:8000). You'll need postgres, redis, celery, and django running.
3. Successfully run PostHog tests: `bin/tests posthog` (which omits Clickhouse tests)
4. Create [your first PR](https://github.com/PostHog/posthog/issues?q=is%3Aissue+is%3Aopen+label%3A%22good+first+issue%22)
and have it be approved. If you work for PostHog someone (Tim or small team lead) will suggest the
first assignment.
## Suggested learning roadmap
1. [Setup your local dev environment](https://posthog.com/docs/developing-locally)
2. Ask your [PostHog Buddy](https://posthog.com/handbook/people/onboarding#posthog-buddy) for a product walk-thru. It's important to get to know the product you are building. I recommend doing this before you become deeply involved in it's internal design. This is a great time to view our product through the eyes of our users.
3. [Review PostHog Project Structure](https://posthog.com/docs/project-structure)
4. Learn [React](https://reactjs.org/docs/hello-world.html), [Redux](https://redux.js.org/introduction/core-concepts), and [Kea](https://kea.js.org/docs/introduction/what-is-kea) - If you're experienced with frontend frameworks I suggest going directly to Kea.
5. Take a brief overview of [Python](https://learnxinyminutes.com/docs/python/).
6. Complete [Django Tutorial 1-5 of 7 parts, skip 6+](https://docs.djangoproject.com/en/3.1/intro/tutorial01/). If you're interested in learning more about Django, pick a copy of [Django book](https://www.feldroy.com/products/two-scoops-of-django-3-x). The company will happily pay for this since they [believe in training us to do our jobs with excellent](https://posthog.com/handbook/people/training). Great place to work, right?
**[Next: Developer Workflow](developer-workflow)**

22
SourceMaterial/engineering/beginners-guide/getting-to-know-posthog.md Normal file
View File

@@ -0,0 +1,22 @@
---
title: 2. Getting To Know PostHog
sidebar: Handbook
showTitle: true
hideAnchor: true
---
It's surprising how enjoyable and calming learning about PostHog's people can be.
You'll find [all their bios here](../../people/team). It's well worth your time!
### PostHog via James Hawkins, CEO
Additionally, James put together some great YouTube videos. I watched them all.
- [Why we built our business in the first place](https://www.youtube.com/watch?v=TIxxIEEvczM)
- [Open Source is Eating SaaS](https://www.youtube.com/watch?v=bh3j_9jVeqg)
- [How we raised a $3M seed round a few weeks after starting our open source project](https://www.youtube.com/watch?v=lJ41-95Ey3w)
- [Open source business models - your choices and how PostHog makes money](https://www.youtube.com/watch?v=L1Ovbzs7vyo)
- [We've never met each other in real life. How we designed PostHog for remote work from day one.](https://www.youtube.com/watch?v=rRwzJiljpSA)
- [Coffee + iPads + Feedback = A day in the of PostHog's graphic designer](https://www.youtube.com/watch?v=xlODCLrZyvM) by Lottie (helpful to see the design side of PostHog)
**[Next: Getting started](getting-started)**

75
SourceMaterial/engineering/beginners-guide/introduction.md Normal file
View File

@@ -0,0 +1,75 @@
---
title: 1. Beginner's Guide
sidebar: Handbook
showTitle: true
hideAnchor: true
---
## Introduction
The *Beginners Guide* started as a project to help me and others get up to speed on PostHog's tech stack.
I also wanted to include bits of advice to make the process encouraging to make working on PostHog even better.
I needed this approach personally since I've been hardcore programming in other languages and tech stacks
for years so most of PostHog's tech stack was newish to me.
## Consider your learning style
Some people like to jump in and go. Some people like to read the docs. I'm the latter. Another way of saying it
that I'm a holistic learner. My plan was to spend part of the time accomplishing work given to me and part
of the time on intentional education. The point is to know your own learning style so that you can be effective.
To that end, if you're a get started now type, you'll appreciate the
[getting started section](getting-started).
It's also important to note that whatever you learn needs to be reviewed several times. It doesn't
matter what learning style you have because review is essential to learning from a neuroscience
perspective. If you create small exercises for yourself such as the projects I've created in this repo
you'll do even better. The good news is that this process doesn't take up an extraordinary amount of time.
## A word about mindset
> You get more than you give.
My mindset is one of wanting to get the most out of life. For me, in part, that
means being excellent and taking full ownership of my career. Learning new
material can be overwhelming not to mention difficult. But remember this:
*it's worth it*! Investing in yourself makes for a better, more capable
version of you. Therefore, trust the difficult process of learning and elevate
your skills. The future you will thank you for your hard work.
If you get discouraged come back to this section.
## What has helped me personally
First thing to remember: starting a new job is like beginning a book in the middle of a
series where the characters are well-formed, and the story is far along. It's fair to say,
you have no idea what is going on. You can read words and understand but not understand
why they are being said. There our goal is to *develop a solid mental model for the codebase*.
> “You don’t care about the answer until you have the question.” - Unknown
- On my first day, I was given an assignment to work on. It began the process of showing me what I needed to learn. Struggling to complete the assignment helped me to *have the question* from the quote above.
- Reading the [project structure doc](/docs/project-structure)
- Browsing the [issues in GitHub](https://github.com/PostHog/posthog/issues) by playing with the various labels. This helped me get a better feel for PostHog's communication style and open-mindedness.
- Pairing with Tim & Eric. They used devtools to examine network traffic, console.log, and I was able to ask specific questions. It was basic stuff and reminded me to use the basic tools I've been using for years. I guess the anxiety of a new job confused my brain a little.
- Reading the kea docs. This is *clutch* to understand the frontend. It's a rather nice library but you won't make progress without understanding Kea.
- Creating a simple app with create-react-app with typescript support `yarn create react-app learn-kea-typescript --template typescript`
> "Take care of yourself. There's no need to burnout in the first month." - Eltje
- Eltje encouraged me to take care of myself, so I did.
> "So, what?!" - [Dare: The New Way to End Anxiety and Stop Panic Attacks](https://www.amazon.com/Dare-Anxiety-Stop-Panic-Attacks/dp/0956596258/) where "D" stands for defuse the anxiety by considering the worst and saying so what.
- Often I felt anxious about my daily contributions. Using the quote above really helps deal with this kind of anxiety. So what if I fail to deliver these assignments timely?! I'm an expert and in time I'll be a great asset to PostHog.
- Additionally, it's important to remember to trust the process. Being new (bad) at something isn't a great feeling initially until you realize that it's a part of the process. Soon you'll be good. It's better to reframe and remember that you only get to have new eyes once. Plus, it's fun to learn new things.
> "When you are working, close your email and slack. No one is watching to see if you are online. In fact, it's the opposite." - Tim, CTO
- Tim told me this on the first or second day, and it was liberating. It allowed me to think of my role as a true `async` open-source contributor. Do what needs to be done so that you can be the most effective.
> Read the docs
- For my learning style, this has been a **must**. I'm keeping a list of resources for learning I've used.
- I also spent time creating projects as you see in this repo which helped me consolidate the knowledge I was gaining.
**[Next: Getting to know PostHog](getting-to-know-posthog)**

32
SourceMaterial/engineering/beginners-guide/notes/django.md Normal file
View File

@@ -0,0 +1,32 @@
---
title: Our Notes On Django
sidebar: Handbook
showTitle: true
hideAnchor: true
---
## Start here
If, like me, you haven't worked with Django before, the best place to start with is
[Writing your first Django app](https://docs.djangoproject.com/en/3.1/intro/tutorial01/)
from the official Django website. This gives you a quick understanding of the major
parts of Django without needing to read an entire book to get it.
## Useful Django commands
- `django-admin startproject mysite` - creates Django project
- `python manage.py runserver` - starts Django web server (optionally add a port at the end `8080`)
- `python manage.py startapp polls` - creates Django app in project
- `python manage.py makemigrations polls` - creates migration scripts in migrations folder
- `python manage.py sqlmigrate polls 0001` - shows SQL that will run for this migration but doesn't perform it
- `python manage.py migrate` - performs all migrations
- `python manage.py shell` - puts you in a Django ORM shell to play with the models on the command-line
- `python manage.py createsuperuser` - creates super user for django admin app which comes by default with all Django projects, url `/admin`
- `python manage.py test polls` - run tests for polls app
## Useful resources
- [Writing your first Django app](https://docs.djangoproject.com/en/3.1/intro/tutorial01/). I recommend
reading Parts 1-5 of the 7 parts, skip 6+ since they are not relevant to PostHog. We do use Django built-in testing so part 5 is required reading.
- [Two Scoops of Django (e-book)](https://www.feldroy.com/products/two-scoops-of-django-3-x)
**[Back: Technologies to learn](../technologies-to-learn)**

103
SourceMaterial/engineering/beginners-guide/notes/docker.md Normal file
View File

@@ -0,0 +1,103 @@
---
title: Our Notes On Docker
sidebar: Handbook
showTitle: true
hideAnchor: true
---
## Docker Nomenclature and Notes
- `Docker Image` - the actual package, **artifact** which can be shared with others, docker images are built in layers via Dockerfile
- `Docker Container` - a *running* instance of a docker image, file system is virtual, contains a port for communication
- Docker run - command which executes *pull* and *start* (only pulls images we do not have locally)
- Docker vs Virtual Machine
- Operating System = Hardware > OS Kernel (layer 1) > Applications (layer 2)
- Docker = Virtualization of applications (layer 2)
- Virtual Image = Virtualization of OS (layer 1)
- Benefits of Docker = images are much smaller, runs faster
- Benefits of VM = you can run different OS (Windows on Linux) since it has it's own OS Kernel
- Docker Port vs Host Port
- Multiple containers may use the same port
- Bind host port to docker port, i.e. host 3000 -> docker 3000, host 3001 -> docker 3000
- `Docker Compose`
- Configuration file specifying *docker commands* to make it easier to work with
- Automatically handles creating a common *docker network*
- Docker compose is usually installed with docker so you already have it
- `Docker Volumes`
- Provides data persistence between host machine and docker containers
- The data between volumes is replicated between the host and docker container volumes
- 3 docker volume types: specified, anonymous, and named volumes, named volumes on the host are managed by docker
- Production should use *named volumes*
- Container Mongodb = /data/db
- Container MySQL = /var/lib/myself
- Container Postgres = /var/lib/postgres/data
- Host Windows = C:\ProgramData\docker\volumes
- Host Linux = /var/lib/docker/volumes/[hash]/_data
- Host Mac = /var/lib/docker/volumes/[hash]/_data
- `screen ~/Library/Containers/com.docker.docker/Data/com.docker.driver.amd64-linux/tty` add access linux VM on mac where data is stored, `ctrl + a + k` to exit screen session
## Basic commands
- `docker pull` downloads a docker image locally
- `docker images` shows a list of local docker images and their sizes
- `docker run` run a docker container, it's two commands in one *docker pull* and *docker start*
- `docker run -d` runs the docker container in *detach mode*
- `docker run -p` binds the container to host port i.e. *-p6000:6370*, *-p [host]:[container]*
- `docker run --name` give the container a name so that you do not need to use the SHA
- `docker run -d -it python` runs python images in *interactive terminal* mode
- `docker run -e` runs an image with an environment variable
- `docker run -net` specify a docker network to run within
- `docker start` start a container, retains the settings from the run command
- `docker stop` - stops a container
- `docker ps` shows *running* containers
- `docker ps -a` shows *running and not-running* containers
- `docker logs` shows the *standard output* of the *running* container
- `docker logs -f` follow the *standard output*, similar to *tail -f*
- `docker exec -it` runs the container with interactive terminal
- `docker network ls` shows a list of the internal docker network
- `docker network create` create a docker network
- `docker build -t my-app:1.0 .` builds an image from a *Dockerfile* in the current directory
- `docker rm` removes a docker container which you need to do before docker rmi
- `docker rmi` removes a docker image, i.e. docker rmi my-app:1.0
- `docker run -v` mounts host filesystem to docker container filesystem
## Docker Compose
- `docker-compose -f mongo.yaml up` pulls, starts, and creates container network
- `docker-compose -f mongo.yaml up -d` runs containers in *detached mode*
- `docker-compose -f mongo.yaml down` stops container, removes container, and stops container network
## First Dockerfile
```docker
FROM python:3.9-alpine3.13
RUN apk add bash nodejs
COPY hello.* /
CMD ["bash"]
```
## First commands
- `docker build .` builds the container
- `docker run --name [name] [sha]` installs the container
- `docker run -it --name [name] [sha]` installs the container and starts in interactive mode
- `docker ps` shows all the running containers
- `docker ps -a` shows all the running and exited containers
- `docker stop [name]` stop container
- `docker start -ai [name]` start container interactively
- `docker rm [name]` removes container
## Resources
- [Creating your first Dockerfile, image and container](https://www.youtube.com/watch?v=hnxI-K10auY) great place to start
- [Docker Tutorial for Beginners [FULL COURSE in 3 Hours]](https://www.youtube.com/watch?v=3c-iBn73dDE) most helpful
- [Docker For Beginners: From Docker Desktop to Deployment](https://www.youtube.com/watch?v=i7ABlHngi1Q)
## Related Resources
- [Kubernetes Tutorial for Beginners FULL COURSE in 4 Hours](https://www.youtube.com/watch?v=X48VuDVv0do) To manage distribution of contains across many servers
**[Back: Technologies to learn](../technologies-to-learn)**

89
SourceMaterial/engineering/beginners-guide/notes/kea.md Normal file
View File

@@ -0,0 +1,89 @@
---
title: Our Notes On Kea
sidebar: Handbook
showTitle: true
hideAnchor: true
---
## Actions
- All code lives inside `logic` which is created with `kea({ ... })`
- Files are typically named `[name]Logic.js|ts`
- `import { useActions } from 'kea'` provides access to action all functions
- All operations start from `actions`
- The mental model for actions is that of *event capturing*, they signal intent
- Sample action: `increment: (amount) => ({ amount })`
- **Actions** map to `reducers` and `listeners` to perform operations
- Actions can invoke several reducers if the name of the action maps to multiple reducers
- Actions defined with `someActions: true` are actions with no arguments
## Reducers
- Reducers define `state` and `operations` on that state.
- `import { useValues } from 'kea'` provides access to the state
- Sample reducers: `counter: [0, { increment: (state, { amount }) => state + amount}]`
- Notice how increment is the same name as the action
- Reducers should never mutate the state directly, they must be pure functions
## Listeners
- Listeners are how you do `side-effects` and async calls to load data
- Listeners may invoke other actions via `actions`, example: `listeners: ({ actions, values }) => ({ ... })`
- Listeners are `async` functions
- Notice we have access to actions and values in the listeners functions
- *Set this or that* is better done by reacting to actions
## Loaders
- Available via the `kea-loaders-plugin`
- Encapsulates the pattern of action > listener > loading > success | failure
- Example: `users: [[], { loadUsers: async () => await api.get('users') }]`
## Selectors
- Selectors handle mapping data across reducers
- Similar to computed values in Vue
## Values
- `import { useValues } from 'kea'`
- You can access values frorm React with useValues or from listeners via listeners function
## Input objects vs functions
- Any of kea's built-in primitives: actions, reducers, listeners, etc. may be declared with an object or function
- Functions are invoked lazily
- Functions are passed 1 argument which can be destructured for actions and values
- Use objects first then functions as you need them
## Props
- Using kea logic as a function allows you to pass in props which are available as destructured props for primitive key functions
## Keyed logic
- If you give your logic a key, you can have multiple independent copies of it. The key is derived from props
- Example: `key: (props) => props.id`
## Breakpoints
- Serves as a debounce function or out of order network calls
## Events
- Handles lifecycle events
## Defaults
- Allows you to specify default values instead of doing them in the reducers
- Defaults may be a function as well to calculate the default values
## Connecting kea logic together
- You may [connect logics together](https://kea.js.org/docs/guide/additional#connecting-logic-together)
## Useful resources
- [Kea](https://kea.js.org/docs/introduction/what-is-kea)
**[Back: Technologies to learn](../technologies-to-learn)**

53
SourceMaterial/engineering/beginners-guide/notes/python.md Normal file
View File

@@ -0,0 +1,53 @@
---
title: Our Notes On Python
sidebar: Handbook
showTitle: true
hideAnchor: true
---
Along with reading about any given programming language it's necessary to use that
knowledge. I've prepared exercises that will help you use the knowledge you are
learning.
Start by reading [Python via Learninyminutes](https://learnxinyminutes.com/docs/python/)
then work to complete the exercises below.
## 1. Fibonacci
You can learn about the [fibonacci here](https://en.wikipedia.org/wiki/Fibonacci_number). Fibonacci
sequence means each number is the sum of the two preceding ones, starting from 0 and 1.
The sequence looks like this `0, 1, 1, 2, 3, 5, 8, 13, 21, 34, 55, 89, 144...`
**Exercise: Calculate the fibonacci sequence up to 100**
## 2. Invictus text processing
1. Read [invictus.txt](https://raw.githubusercontent.com/buwilliams/learn-posthog/main/docs/exercises/02_invictus.txt) into a string
2. Split the string an array of words and print them out
3. Correct words with invalid characters and print the cleaned words out
4. Create an array of maps of all unique words and how many times they occurred
5. Sort the array of maps by the number of times they occurred in descending order
6. Convert the code into a class, implement interfaces and type checking if applicable
## More exercises
- [Programming Problems](https://adriann.github.io/programming_problems.html)
## Create your own exercises
One interesting strategy is to:
1. Buy/find a programming book you're interested in.
2. As your read, create an exercise for that chapter.
3. Code the exercise that you created before moving on.
4. Rinse and repeat until you've finished the book.
By creating the exercise and solving it, you'll learn better than if you just read the chapter.
Remember that many programmers are lazy and are unwilling to put this kind of effort. You'll
be light years ahead of your peers as you apply yourself.
## Useful resources
- [Python via Learninyminutes](https://learnxinyminutes.com/docs/python/)
**[Back: Technologies to learn](../technologies-to-learn)**

116
SourceMaterial/engineering/beginners-guide/notes/react.md Normal file
View File

@@ -0,0 +1,116 @@
---
title: Our Notes On React
sidebar: Handbook
showTitle: true
hideAnchor: true
---
The [React docs](https://reactjs.org/docs/getting-started.html) are great for getting from zero to one.
## Hooks
I found hooks somewhat counterintuitive at first, but they're very powerful once you grasp them. Refer to the [Rules of Hooks](https://reactjs.org/docs/hooks-rules.html). - @samwinslow
1. Only call from the top level of a functional component
2. Do not call outside a functional component or from plain JS (you can call from custom hooks)
### useState
Uses destructured array assignment syntax
`const [value, setValue] = useState(initialValue)`
An updater function can be passed to the setter so that multiple updates can be called in sequence, or to merge-update the state via spreading if it's an object. The updater is a pure function which takes previous state and returns next.
```jsx
// bad
setValue(value + 1)
setValue(value + 1)
// good
setValue(value => value + 1)
setValue(value => value + 1)
```
In general, derive data while rendering rather than storing derived values in state (e.g. filtering local data). However, if expensive filtering or join operations are to be performed and/or the component re-renders frequently, a memoized state management library might be better.
### useEffect
Takes a callback function which may have (potentially global) side effects. Runs on every re-render by default.
```jsx
function EffectExample() {
const [value, setValue] = useState(initialValue)
useEffect(() => {
document.title = `The value is now ${value}`
})
return (
<div>
<p>{value}</p>
<button onClick={() => setValue(value => value + 1)}>Use Effect</button>
</div>
)
}
```
The rendered value is not a special data binding that causes it to listen. It is merely a reflection of a new value rendered as a result of calling the setter.
Can return a cleanup function from the effect and declare when it should run
```jsx
function ApiStatus({ service }) {
const [isOnline, setOnline] = useState(null)
const { id } = service
useEffect(() => {
const handleStatusChange(status) => {
setOnline(status.isOnline)
}
SomeApi.subscribe(id, handleStatusChange)
return () => SomeApi.unsubscribe(id, handleStatusChange)
}, [id]) // Only run when `id` changes (sync to state)
// rendering
}
```
### useLayoutEffect
Same as `useEffect`, but runs callback synchronously during commit lifecycle phase
### useMemo
Recalculates value only when dependencies change
### useCallback
Updates callback function reference when dependencies change
### useRef
Mutable ref used to access returned child.
- Persists between renders
- Default: `{ current: null }`
- Plain object; mutating does not trigger re-renders
### Custom Hooks
Listeners and API connections can be extracted to a custom hook and reused
Examples from popular libraries:
- React-Redux: `useSelector`, `useDispatch`
- React-Router: `useHistory`, `useLocation`, `useParams`
- `useFormState`
## Useful resources
- [Dan Abramov - A Complete Guide to useEffect](https://overreacted.io/a-complete-guide-to-useeffect/)
- [Mark Erikson - A Complete Guide to React Rendering Behavior](https://blog.isquaredsoftware.com/2020/05/blogged-answers-a-mostly-complete-guide-to-react-rendering-behavior/)
**[Back: Technologies to learn](../technologies-to-learn)**

23
SourceMaterial/engineering/beginners-guide/notes/typescript.md Normal file
View File

@@ -0,0 +1,23 @@
---
title: Our Notes On TypeScript
sidebar: Handbook
showTitle: true
hideAnchor: true
---
The best way to learn TypeScript is to read introductory material then get hands on with exercises.
## Exercises
You can complete the exercises on the [TypeScript Playground](https://www.typescriptlang.org/play)
You can use the same [exercises for Python](python) as starting place.
## Great places to learn
- [TypeScript via learnxinyminutes](https://learnxinyminutes.com/docs/typescript/)
- [TypeScript in 5 minutes](https://www.typescriptlang.org/docs/handbook/typescript-in-5-minutes.html)
- [TypeScript Handbook](https://www.typescriptlang.org/docs/handbook/intro.html)
**[Back: Technologies to learn](../technologies-to-learn)**

33
SourceMaterial/engineering/beginners-guide/technologies-to-learn.md Normal file
View File

@@ -0,0 +1,33 @@
---
title: 5. Technologies To Learn
sidebar: Handbook
showTitle: true
hideAnchor: true
---
Each of the links below will send you to the best resources for learning I found. As I went through learning the
various technologies, I took notes. Below you'll find all the notes that I took along with resources I felt were
particularly useful. I'd suggest using *our notes* links as you would a cheatsheet.
## Backend
- [Python](https://www.python.org/) ([our notes on Python](notes/python))
- [Django](https://www.djangoproject.com/) ([our notes on Django](notes/django))
- [Django Testing](https://docs.djangoproject.com/en/3.1/intro/tutorial05/)
- [Pytest](https://docs.pytest.org/en/stable/getting-started.html)
- [Clickhouse](https://clickhouse.tech/) (enterprise database)
- [Celery](https://docs.celeryproject.org/en/stable/) (we use Redis as Celery's message broker)
- [Docker](https://www.docker.com/) ([our notes on Docker](notes/docker))
## Frontend
- [React](https://reactjs.org/docs/hello-world.html) ([our notes on React](notes/react))
- [Redux](https://redux.js.org/introduction/core-concepts)
- [Kea](https://kea.js.org/docs/introduction/what-is-kea) ([our notes on Kea](notes/kea))
- [TypeScript](https://www.typescriptlang.org/) ([our notes on TypeScript](notes/typescript))
## Useful tech
- [Tmux](https://github.com/tmux/tmux/wiki)
- [Fish](https://github.com/fish-shell/fish-shell)
- [Zsh](https://github.com/ohmyzsh/ohmyzsh)

57
SourceMaterial/engineering/bug-prioritization.md Normal file
View File

@@ -0,0 +1,57 @@
---
title: Bug Prioritization
sidebar: Handbook
showTitle: true
---
## User experience degradation
When bugs are reported it's critical to properly gauge the extent and impact to be able to prioritize and respond accordingly. These are the priorities we use across the entire engineering org, along with the relevant labels to quickly identify them in GitHub.
> Please always remember to tag your issues with the relevant priority.
<span>
<table>
<tr>
<td>GitHub Label</td>
<td>Description</td>
</tr>
<tr>
<td><span class="tag-label" style="background:#ff0000; color: white;">P0</span></td>
<td>Critical, breaking issue (page crash, missing functionality)</td>
</tr>
<tr>
<td><span class="tag-label" style="background:#f0a000;">P1</span></td>
<td>Urgent, non-breaking (no crash but low usability)</td>
</tr>
<tr>
<td ><span class="tag-label"style="background:#ffe000;">P2</span></td>
<td>Semi-urgent, non-breaking, affects UX but functional</td>
</tr>
<tr>
<td><span class="tag-label" style="background:#1d76db; color: white;">P3</span></td>
<td>Icebox, address when possible</td>
</tr>
</table>
</span>
## Security issues
Security issues, due to their nature, have a different prioritization schema. This schema is also in line with our internal SOC 2 related policies (Vulnerability Management Policy). When filing security-related GitHub issues, remember to attach label `security` and the appropriate priority label. More details on filing can be found in the [README](https://github.com/PostHog/product-internal#README) of the `product-internal` repo.
<blockquote class="warning-note">
Security issue information should not be made public until a fix is live and sufficiently (ideally completely) adopted.
</blockquote>
PostHog security issues include a priority (severity) level. This level is based on our self-calculated CVSS score for each specific vulnerability. CVSS is an industry standard vulnerability metric. You can learn more about CVSS at [FIRST.org](https://www.first.org/cvss/user-guide) and calculate it using the FIRST.org [calculator](https://www.first.org/cvss/calculator/3.1).
| GitHub Label | Priority Level | CVSS V3 Score Range | Definition | Examples |
|---|---|---|---|---|
|**security-P0**|Critical|9.0 - 10.0|Vulnerabilities that cause a privilege escalation on the platform from unprivileged to admin, allows remote code execution, financial theft, unauthorized access to/extraction of sensitive data, etc.|Vulnerabilities that result in Remote Code Execution such as Vertical Authentication bypass, SSRF, XXE, SQL Injection, User authentication bypass|
|**security-P1**|High|7.0 - 8.9|Vulnerabilities that affect the security of the platform including the processes it supports.|Lateral authentication bypass, Stored XSS, some CSRF depending on impact|
|**security-P2**|Medium|4.0 - 6.9|Vulnerabilities that affect multiple users, and require little or no user interaction to trigger.|Reflective XSS, Direct object reference, URL Redirect, some CSRF depending on impact|
|**security-P3**|Low|0.1 - 3.9|Issues that affect singular users and require interaction or significant prerequisites (MitM) to trigger.|Common flaws, Debug information, Mixed Content|

13
SourceMaterial/engineering/common-issues.md Normal file
View File

@@ -0,0 +1,13 @@
---
title: Common Issues
sidebar: Handbook
showTitle: true
---
A page to host big issues raised by users (rather than small bugs) that we have certain answers for.
### Shopify Event-Logging "Bug"
Admin users who have PostHog setup for Shopify may experience their events being logged for another user.
This is not an issue with PostHog. Rather, this is due to a feature Shopify offers where it stores all the session data and lets you browse your website as if you were your client with all their cookies and local/session storage.

75
SourceMaterial/engineering/development-process.md Normal file
View File

@@ -0,0 +1,75 @@
---
title: Development Process
sidebar: Handbook
showTitle: true
---
> _**Note:** This guide is aimed at people who work for PostHog. If you want to contribute, [see our Contributing Guide](/docs/contributing)._
<br />
Any process is a balance between speed and control. If we have a long process that requires extensive QA and 10 approvals, we will never make mistakes because we will never release anything.
However, if we have no checks in place, we will release quickly but everything will be broken.
## 1. How to Decide What to Build
There are 3 places that work comes from.
- Issues/bugs (raised by the community or us)
- [Roadmap](/handbook/strategy/roadmap)
- "This should be better"
## 2. Sizing a Task
When picking up a task, it should be doable in a day, including code review and QA. If it's not, you need to break it down into smaller chunks until it is. Tasks of this size are easy to test, easy to deploy, less likely to cause merge conflicts, and should still deliver some kind of value.
Even if you're contributing, this is helpful as it means you'll be able to contribute to PostHog faster.
## 3. Writing Code
We're big fans of Test Driven Development (TDD). We've tried to create test infrastructure that helps you rather than annoys you. If that isn't the case, please raise an issue! Keeping tests on point is a high priority to keep developer productivity high.
Other than that, you know what to do in this section.
## 4. Creating a PR
To make sure our issues are linked correctly to the PRs, you can tag the issue in your commit.
```bash
git commit -m "Closes #289 add posthog logo to website"
```
## 5. Code Review
When we review a PR, we'll look at the following things:
- Does the PR actually solve the issue?
- Does the solution make sense?
- Will the code perform with millions of events/users/actions?
- Are there tests and do they test the right things?
- Are there any security flaws?
Things we do not care about during review:
- Syntax. If we're arguing about syntax, that means we should install a code formatter
See: [How we review](/handbook/engineering/how-we-review).
## 7. Merging
Merge anytime. Friday afternoon? Merge.
Our testing, reviewing and building process should be good enough that we're comfortable merging any time.
## How to Test
See: [How to test](/docs/contributing#testing).
## How we Review
See: [How we review](/handbook/engineering/how-we-review).
## How to release a new version
See: [Release new version](/handbook/engineering/release-new-version).

46
SourceMaterial/engineering/how-we-review.md Normal file
View File

@@ -0,0 +1,46 @@
---
title: How We Review PRs
sidebar: Handbook
showTitle: true
---
Almost all PRs made to PostHog repositories will need a review from another engineer. We do this because, almost every time we review a PR, we find a bug, a performance issue, unnecessary code or UX that could have been confusing.
## How to review
1. Have a flick through the code.
- What to look for:
- Code that does the wrong thing or will produce bugs
- Code that you think will cause performance issues
- Are there tests for all of the new functionality, and do they test the right thing?
- Any security issues or project leakage?
- Is the code properly instrumented to allow tracking of every relevant action (i.e. all the relevant frontend elements have unique and helpful `data-attr`s, and there are backend events where appropriate)?
- What _not_ to look for:
- Formatting issues (prettier should handle this, raise a PR to fix that)
- "I would have done it differently" (Unless the code is completely incomprehensible or unreadable, or will cause us massive harm long term - as long as it works, it's good enough.)
2. Open the review app or check the branch out locally.
- What to look for:
- Bugs in the new functionality (if you're reviewing the insights page, make sure you try breakdown, cohorts, filters, different time frames etc)
- Confusing UX
- Confusing wording
- Backend tracked events not being fired properly or with an incorrect payload.
- Should the code be behind a feature flag?
- If the code is behind a feature flag, do all cases work properly? (in particular, make sure the old functionality does not break)
- Are we building the right thing? (We should be willing to throw away PRs or start over)
- Don't be shy here - try to break it!
- What not to look for:
- Issues _not_ related to this PR. Create a new issue for those.
The emphasis should be on getting something out quickly. Endless review cycles sap energy and enthusiasm.
## Setting up Heroku Test Environment
1. Go to the pull request you want to QA on.
2. Go to the Heroku test environment.
Check/do the following:
1. If the environment was already deployed, it should say "This branch was successfully deployed"
1. If it says ‘This branch was not deployed’ go to the Heroku pipeline and click ‘Create review app’
1. If the PR was submitted from a fork, you'll need to test the changes locally.
3. Open the app, create a new account, and start testing!

54
SourceMaterial/engineering/onboarding-customer.md Normal file
View File

@@ -0,0 +1,54 @@
---
title: Onboarding Customers
sidebar: Handbook
showTitle: true
---
A lot of people want to set up PostHog on their own without talking to anyone, and we'll make sure that path is as optimised as possible. However, sometimes it's more efficient/better to talk to one of us over a call.
Our user notes are not shared publicly. [Read them here](https://docs.google.com/document/d/1gJlsUDrlW7ur8zT5scqRvXZhapm_0JdvKGiw68Iyx9E/edit#heading=h.q9lg9hgl34g2).
A standard structure you could maintain while doing that call is:
## 1. Introductions / Find Out About Their Business
1. Make sure you introduce yourself and check it's ok if you take notes.
1. Introduce PostHog: "We're open-source product analytics. Think Mixpanel or Amplitude, but with full control over your data and focused on engineers"
1. Ask questions about their business:
- Why did they reach out in the first place?
- What are their main goals/most important metric for this quarter/batch/year?
- Are they familiar with other product analytics tools? Are they using any right now?
- What stage is their company at - do they have users, are they about to launch, are they trying to expand?
- How does their app work?
- Where does the app live? Website, online app, mobile app, backend?
## 2. Demo
Share your own screen and wizz through a demo of PostHog. The following order tends to work best:
1. Tell them the big picture of how the demo will work "I am going to show you two main things - how we collect data and what we help you do with it"
1. Go to /events, show raw events coming in and explain how we're different from Mixpanel/Amplitude as we track all clicks, which means no `posthog.track()` calls necessary
1. Go to /trends, show filtering by url, DAU, breakdown by initial domain referrer, anything else they ask for. It's a good opportunity to share some knowledge e.g. how to understand where traffic is coming from (UTM tags or referring domain)
1. Go to /funnels, explain how those work and that PostHog specifically allows you to see each user that goes through the funnel, rather than aggregates
1. Go to /people/retention and explain how the Retention table works
1. Show off the Toolbar using the "Launch Toolbar" button
1. Explain Feature Flags
1. Go to Default Dashboard
1. Optional: show paths
1. Ask if any questions
## 3. Setup
Flip it around and ask them to share their screen.
0. If they haven't setup PostHog yet, walk them through and help them install the snippet on their website (and anywhere else necessary)
1. Based on the questions from Section 1, set up relevant dashboards for them.
- DAUs are already there, but they might want a /trends view split out by different pages. Have them add that to default dashboard. They may also want to see where traffic is coming from as a separate item in trends to add to the dashboard.
- Have them create an action for a click on their CTA on the home page
- Create a funnel with two steps: pageviews and the CTA action
1. (Optional) if they have an app, help them set up `identify` calls correctly
1. Ask if there any questions
## 4. Conclusion
1. Ask them to join our Slack group (and send an email right after!)

58
SourceMaterial/engineering/release-new-version.md Normal file
View File

@@ -0,0 +1,58 @@
---
title: Releasing a New Version
sidebar: Handbook
showTitle: true
---
At the moment, we release a new version every two weeks ([unless it makes sense not to!](/blog/we-ship-whenever)). This might change in the future.
## Version Numbers
Every week we up the 'minor' in `major.minor.patch`. At the moment, we're at version 1 for major. This will only change once we have released sufficient functionality under stage 2 of [our Roadmap](/handbook/strategy/roadmap/).
Hopefully we will not have to do many patch versions, but if between versions we discover a breaking bug, we will.
## Timeline
Three days before we release, on Monday, we institute a code freeze. We branch master into release-[version] and deploy that to our production environment (app.posthog.com). Only bugfixes are allowed to be merged into this branch (and thus put on production) between Monday and the release going out. This gives us about three days to test if this release has any bugs.
## Checklist
<input type="checkbox"/> Figure out what's updated in this release
- `git checkout release-[version]`
- `git log --pretty=format:%s [old-version]..head`
<br />
<input type="checkbox"/> Write up the PostHog Array [blog post](/handbook/growth/marketing/posthog-array)
<input type="checkbox"/> Copy from PostHog Array and write up the changes into `CHANGELOG.md` following the structure of the previous release
- `git add CHANGELOG.md`
- `git commit -m "Changelog version 1.7.0"`
<br />
<input type="checkbox"/> Update the `VERSION` in `posthog/version.py`
- `git checkout release-[version]`
- `git add posthog/version.py`
- `git commit -m "Bump version [version]"`
<br />
<input type="checkbox"/> Tag the version
- `git tag -a [version] -m "Version [version]"`
- `git push origin head --tags`
Once a new Docker image has been built (see [Docker Hub](https://hub.docker.com), password in 1password) for the new version, open the [charts](https://github.com/PostHog/charts) repo and make the changes:
1. Edit the **two** Chart files: [Chart.yaml](https://github.com/PostHog/charts/blob/master/charts/posthog/Chart.yaml) and [ChartV3.yaml](https://github.com/PostHog/charts/blob/master/charts/posthog/ChartV3.yaml), in both:
- Bump `appVersion` to the latest app version (same number as on the docker image).
- Bump `version` (chart version) patch number, unless making big changes to the chart itself. Lesson learned: this can only be `x.x.x`. It can't have a fourth part.
2. Change the docker tag in [values.yaml](https://github.com/PostHog/charts/blob/master/charts/posthog/values.yaml#L6) to point to [the latest tag](https://hub.docker.com/r/posthog/posthog/tags?page=1&ordering=last_updated).
3. `git commit -m 'Bump PostHog app version to 1.0.XX, release chart version 1.0.YY'`
4. `git tag -a 1.0.YY -m "Version 1.0.YY"`
5. `git push && git push origin head --tags`
Finally to bump the `latest-release` docker image, log to [hub.docker.com](https://hub.docker.com/repository/docker/posthog/posthog/builds) and configure a new automatic build. Set the docker tag to `latest-release` and the source to the tag `1.XX.YY`. Delete any older tag with the same name if present and click "save & build"

12
SourceMaterial/engineering/releasing-python.md Normal file
View File

@@ -0,0 +1,12 @@
---
title: Releasing a New Version (Python Library)
sidebar: Handbook
showTitle: true
---
## How to Release
1. Increase `VERSION` in `posthog/version.py`
2. Update `CHANGELOG.md` with a short summary of the changes
3. run `make release` and `make release_analytics`
4. `git commit -am "Release X.Y.Z."` (where X.Y.Z is the new version)
5. `git tag -a X.Y.Z -m "Version X.Y.Z"` (where X.Y.Z is the new version).

137
SourceMaterial/engineering/setup-ssl-locally.md Normal file
View File

@@ -0,0 +1,137 @@
---
title: Setup SSL locally
sidebar: Handbook
showTitle: true
---
Setting up HTTPS locally can be useful if you're trying to debug hard
to replicate issues (e.g cross domain cookies, etc).
There are two ways you can get HTTPS locally:
1. ngrok
2. NGINX and a local certificate.
The easiest option is to use ngrok.
## Set up SSL via ngrok
1. Make sure you [have ngrok installed](https://ngrok.com/download).
2. Sign up for an ngrok account (or sign in with GitHub) and run `ngrok authtoken <TOKEN>`
3. Edit `$HOME/.ngrok2/ngrok.yml` and add the following after the line with `authtoken: <TOKEN>`:
```
tunnels:
django:
proto: http
addr: 8000
webpack:
proto: http
addr: 8234
```
4. Start ngrok. This will give you tunnel URLs such as https://68f83839843a.ngrok.io
```bash
ngrok start --all
```
5. Copy the HTTPS URL for the tunnel to port 8234 and set it as the value for the `JS_URL` environment variable. Then, start webpack:
```bash
export WEBPACK_HOT_RELOAD_HOST=0.0.0.0
export LOCAL_HTTPS=1
export JS_URL=https://68f83839843a.ngrok.io
yarn start
```
6. Use the same URL as the value for `JS_URL` again and start the Django server
```bash
export DEBUG=1
export LOCAL_HTTPS=1
export JS_URL=https://68f83839843a.ngrok.io
python manage.py runserver
```
7. Open the HTTPS URL for the tunnel to port 8000.
**Tips & Tricks**
If you're testing the Toolbar, make sure to add the ngrok urls to the list on the 'Project Settings' page.
![Permitted domains](../../images/engineering/toolbar-permitted-ngrok.png)
Also, watch out, network requests can be slow through ngrok:
![Network slow with ngrok](../../images/engineering/ngrok-slow.gif)
## Set up SSL via NGINX and a local certificate
0. Update openssl if "openssl version" tells you "LibreSSL" or something like that.
In case `brew install openssl` and `brew link openssl` don't work well, use
`/usr/local/opt/openssl/bin/openssl` instead of `openssl` in the next step.
1. Create key
```
openssl req -x509 -newkey rsa:4096 -sha256 -days 3650 -nodes \
-keyout localhost.key -out localhost.crt -subj "/CN=secure.posthog.dev" \
-addext "subjectAltName=DNS:secure.posthog.dev,IP:10.0.0.1"
```
2. Trust the key for Chrome/Safari
```
sudo security add-trusted-cert -d -r trustRoot -k /Library/Keychains/System.keychain localhost.crt
```
3. Add `secure.posthog.dev` to /etc/hosts
```
127.0.0.1 secure.posthog.dev
```
4. Install nginx (`brew install nginx`) and add the following config in `/usr/local/etc/nginx/nginx.conf`
```nginx
upstream backend {
server 127.0.0.1:8000;
}
server {
server_name secure.posthog.dev;
rewrite ^(.*) https://secure.posthog.dev$1 permanent;
}
server {
listen 443 ssl;
server_name secure.posthog.dev;
ssl_certificate /Users/timglaser/dev/localhost.crt;
ssl_certificate_key /Users/timglaser/dev/localhost.key ;
ssl_prefer_server_ciphers on;
ssl_session_cache shared:SSL:1m;
ssl_session_timeout 5m;
ssl_ciphers HIGH:!aNULL:!MD5;
location / {
proxy_pass http://backend;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header Host $http_host;
proxy_redirect off;
proxy_set_header X-Forwarded-Proto $scheme;
}
location /static/ {
proxy_pass http://127.0.0.1:8234/static/;
}
}
```
5. Add the following command to start nginx
```bash
nginx -p /usr/local/etc/nginx/ -c /usr/local/etc/nginx/nginx.conf
```
6. You can stop the nginx server with
```bash
nginx -p /usr/local/etc/nginx/ -c /usr/local/etc/nginx/nginx.conf -s stop
```
7. To run local development, use
```bash
bin/start-http
```