diff --git a/.dockerignore b/.dockerignore index 5192e3b..2edfb47 100644 --- a/.dockerignore +++ b/.dockerignore @@ -32,5 +32,6 @@ Thumbs.db .venv # Local Developer Notes -dev dist +dev +.cache-trivy diff --git a/.gitignore b/.gitignore index 1521c8b..650294f 100644 --- a/.gitignore +++ b/.gitignore @@ -1 +1,3 @@ dist +dev +.cache-trivy diff --git a/Dockerfile b/Dockerfile index e5a7021..be70c3a 100644 --- a/Dockerfile +++ b/Dockerfile @@ -16,4 +16,6 @@ ENV SERVER_PORT 8787 ENV SERVER_ROOT /app ENV SERVER_LOG_LEVEL info +USER 5020 + LABEL maintainer="s174509@dtu.dk" diff --git a/README.md b/README.md index b1bbdef..48107da 100644 --- a/README.md +++ b/README.md @@ -1,3 +1,128 @@ -# TODO -- [ ] Don't use commit ID when tagging dev containers / check clean git tree before allowing a commit ID tag. -- [ ] `pre-commit install` as part of `run.sh`? +# DTU Python Support +This is the repository containing the "DTU Python Support" book. +See it live: https://pysupport.timesigned.com. + +## Getting Started +We've automated all the gnarly steps between beautiful `*.md` source files, and a real, running website. + +You'll need to have the following commands available: +- `git`: +- `python`: Newer than `3.9`. +- `podman`: +- `pre-commit`: +- `echo "I am not afraid of the terminal!"` + +Once that's set, go ahead and grab (`clone`, in `git` terms) this repo: +```bash +git clone https://git.sofus.io/so-rose/site-support.git +cd site-support +``` + +Now, you can view the book in your local browser: +```bash +./run.py app +## Now, navigate to http://localhost:8787 in your browser. +## CTRL+C to exit. +``` + +Run this whenever you want to preview any changes you made to `src/*.md` files! + + + +# Contributing +Let's say you've made some cool changes, and you want to propose they go into the book. +First of all, thank you! +But, how do? + +At this point, you'll need to know a bit of `git`. +Incidentally, the book contains a quick guide at [`src/basics/collaboration/git`](./src/basics/collaboration/git)! +This git-powered puzzle game is also a surprisingly effective teacher: + +Armed with this knowledge, make a branch **within the repo folder**: +```bash +git checkout -b +git branch +``` +You're now on a new branch, which belongs only to you! + +Now, commit your changes. Remember to use [Conventional Commit](https://www.conventionalcommits.org/en/v1.0.0/) format for your commit message. +For example: +```bash +git add -A +git status ## Always review your changes! +git commit -m "feat: New page!" +``` +That's it! Give it a `git push`. + +All that's left is to start a [Pull Request](https://git.sofus.io/so-rose/site-support/pulls), and wait for someone on the team to examine your contribution. + +When several people collaborate, the workflow might end up looking something like this: +```mermaid +gitGraph + commit + commit tag: "v0.1.0" + branch bobs-cool-new-feature + branch jills-hotfix + checkout bobs-cool-new-feature + commit + commit + commit + checkout jills-hotfix + commit + commit + checkout main + merge jills-hotfix tag: "v0.1.1" + merge bobs-cool-new-feature tag: "v0.2.0" +``` + + + +# Structure of the Book +The book is structured to minimize technical wonk, and let you *focus on writing*. +This is made possible due to `mdbook`, this project avoids a lot of the technical boiler + +## Easy Overview / ToC / Menu: `src/SUMMARY.md` +The beating heart of the book is [`src/SUMMARY.md`](./src/SUMMARY.md). +The headings, bullet nesting, and links result in the side-menu. + + +## Written in Markdown: A Gentler Take on Text +"Markdown" is text for humans. That's it! + +This **entire book** is written in **100% normal** "Markdown". +The *only* filetype in `src/` is `*.md`. + +All `.md` files can be opened in *any text editor*. +No, not Word; think Notepad, vim, VSCodium! + +But what does it look like? Ya know, `.md` files? + +```markdown +It looks pretty normal, honestly. +You can **bold**, *italic*, and even write `cool_l33t_code`! + +You'll need a [text editor of some kind](https://neovim.io/), of course. +But you'll find that super quick; after all, you are: +- Cool +- Knowledgeable +- Awesome +``` + +With a little bit of extension magic, like the built-in MermaidJS, you can even impress your cat! +```mermaid +journey + title Learning to Use site-support: A User Journey + section Utter Confusion + Find the Repo: 1: You, A Rus, A Kitten + Clone the Repo: 3: You, A Kitten + Notice it's all .md: 5: You + section Fun and Games + Discover ./run.py app: 7: You + Change stuff: 9: You +``` + + + +# Conclusion +You should have everything you need to get started. +If anything is still confusing, don't hesitate to report an [Issue](https://git.sofus.io/so-rose/site-support/issues)! diff --git a/run.py b/run.py index a7a1fd4..47d4b53 100755 --- a/run.py +++ b/run.py @@ -40,11 +40,11 @@ CMD_DEPENDENCIES = [ ] IMAGE_NAME = "site-support" -IMAGE_VERSION = ("0", "0", "1") +IMAGE_VERSION = ("0", "1", "0") REGISTRY_HOST = "git.sofus.io" REGISTRY_USER = "so-rose" -REGISTRY_PASSWORD = subprocess.check_output( +REGISTRY_PASSWORD = lambda: subprocess.check_output( ['pass', 'services/home/git.sofus.io/container-registry-token'] ).decode('ascii').strip() @@ -52,18 +52,28 @@ REGISTRY_PASSWORD = subprocess.check_output( # - Help Text #################### def action_help() -> None: - print("""This script provides one-click development, CI, and deployment. + print(f"""This script provides one-click development, CI, and deployment, + to support the use of the {IMAGE_NAME} project. Usage: echo -e "./run.py [OPTION] [EXTRAS]" + + Program Information: + Version + => {".".join(IMAGE_VERSION)} + OCI Container + => {REGISTRY_HOST}/{REGISTRY_USER}/{IMAGE_NAME}:{IMAGE_VERSION[0] The following commands must be available: podman - => The project is developed and run in podman containers. + => This project is developed and run in podman containers. + => https://podman.io/ git => This project uses git for versioning, and collaboration. + => https://git-scm.com/ pre-commit => Enforces that certain checks pass at each commit. + => https://pre-commit.com/ Options: ./run.py @@ -74,40 +84,41 @@ def action_help() -> None: Options, Run Locally: ./run.py app - => Will run the app on port 8787, for local development. + => Will run the app live on port 8787. Options, Check: ./run.py check => Will run all checks, including static analysis and tests. ./run.py analyze-quality [OPTIONS] - => Will run the static code-quality analysis suite. - => OPTIONS will be passed directly to the tool (ruff). + => Not yet implemented... + => Potentially, could run ruff on Python snippets in the book. ./run.py analyze-types [OPTIONS] - => Will run the static type checking suite. - => OPTIONS will be passed directly to the tool (mypy). + => Not yet implemented... + => Potentially, could run mypy on Python snippets in the book. ./run.py analyze-security => Will run the static security analysis suite. ./run.py analyze-format [--overwrite] [OPTIONS] - => Will run the code formatter in read-only omde. - => '--overwrite' will cause the formater to reformat all files. - => OPTIONS will be passed directly to the tool (tan). + => Not yet implemented... + => Potentially, could format Python snippets in the book. ./run.py test + => Not yet impelmented... + => Potentially, would test distro commands in an appropriate container. + => Potentially, would turn Python snippets into pytest units. + - See https://github.com/modal-labs/pytest-markdown-docs => Will run all Markdown Python snippets as tests. Options, Build & Deploy: ./run.py build - => Will build, tag and upload a docker image appropriately. - - ./run.py build-release - => Will build, tag and upload a docker image appropriate for release. + => Will build a docker image, and tag it :dev. Options, Housekeeping: ./run.py clean + => Not yet implemented... => Will delete all data caused by this project's presence on your system. """) @@ -161,15 +172,13 @@ def action_publish() -> None: subprocess.run([ "podman", "login", REGISTRY_HOST, "--username", REGISTRY_USER, - "--password", REGISTRY_PASSWORD, + "--password", REGISTRY_PASSWORD(), ]) - ## TODO: Ensure git tag matches. - # Tag & Publish Image @ : subprocess.run([ "podman", "image", "push", - f"{IMAGE_NAME}:dev", + f"{IMAGE_NAME}:{get_git_revision_hash()}", f"{REGISTRY_HOST}/{REGISTRY_USER}/{IMAGE_NAME}:{get_git_revision_hash()}", ]) @@ -182,7 +191,7 @@ def action_publish() -> None: # Tag Image subprocess.run([ "podman", "tag", - f"{IMAGE_NAME}:dev", + f"{IMAGE_NAME}:{get_git_revision_hash()}", image_tag ]) @@ -206,6 +215,22 @@ def action_app() -> None: ]) +#################### +# - Actions - Analyze +#################### +def action_analyze_security() -> None: + Path('.cache-trivy').mkdir(parents=True, exist_ok=True) + subprocess.run([ + "podman", "run", "--rm", "-it", + "--workdir", "/src", + "--volume", ".:/src", + "--volume", "./.cache-trivy:/root/.cache", + "ghcr.io/aquasecurity/trivy:latest", + "fs", "--quiet", "--scanners", "vuln,secret,config,license", + "--exit-code", "1", "./" + ]) + + #################### @@ -215,7 +240,7 @@ if __name__ == "__main__": # Check Available Commands for cmd in CMD_DEPENDENCIES: if not cmd_exists(cmd) : - print("One or more dependencies are not installed. Please see --help.") + print(f"{cmd} is not installed. Please see --help for instructions.") sys.exit(1) with cd_script_dir(): @@ -227,6 +252,8 @@ if __name__ == "__main__": "app": action_app, "dev": lambda: print("TBD"), + "analyze-security": action_analyze_security, + "help": action_help, "-h": action_help, "--help": action_help, diff --git a/src/basics/collaboration/git.md b/src/basics/collaboration/git.md index e69de29..9789688 100644 --- a/src/basics/collaboration/git.md +++ b/src/basics/collaboration/git.md @@ -0,0 +1,66 @@ +# For git noobsies +Start by reading: http://rogerdudler.github.io/git-guide/. + +This is also not meant to be an in depth introduction to git. It should, however, be enough to get you working. Also, it assumes you have a working git installation. + +To **configure** git: + 1. `git config --global user.name ` where `` is your git (or GitHub) username. + 2. `git config --global user.email ` where `` is your git (or GitHub) email. + 3. Confirm that the information is correct with `git config --global --list`. + +To **clone** (download) this repository: + 1. Move to the desired directory and write `git clone https://github.com/volesen/DiscreteMath.git ` where `` is the name of the directory to place it in. + +To **pull** (update your local repository) from origin: + 1. `git pull`. + 2. Done. If this fails, it's probably because you've changed or removed some files others have also changed or removed. + +To create a **branch**: + 1. Create an issue using the web GUI. From here on `` will refer to the id of the issue. For example 14. + 1. Create a new branch `git checkout -b ` where `` is `-issue`. This also moves your HEAD to the new branch. + +To **push** (upload) your changes to the current branch: + 1. Before anything else, make sure to pull the current branch with `git pull`. This helps in dealing with merge conflicts + 2. `git add .` to add all your changed, deleted and added files to the "staging area". + 3. `git commit -m ""` to create a "commit" with all your changes. This only changes things in your local repository. should be a short description of what your commit changes. [A great guide to writing good messages](https://chris.beams.io/posts/git-commit/). + 4. `git push` to push your newly created commit to the branch. This is the first time any changes are made outside of your computer. [branch](https://git-scm.com/book/en/v2/Git-Branching-Basic-Branching-and-Merging). If it is a new branch write `git push --set-upstream origin ` where `` is the name of the branch. + 5. Aaand your done. + +To **swap** branch (if you want to work at multiple branches simultaneously): + 1. `git checkout ` where `` is the name of the branch. + +To **delete** a branch: + 1. To delete a local branch write `git branch -d ` where `` is the name of the branch. + 1. To delete a remote branch (be careful!) write `git push origin --delete ` where is the name of the branch. + +To **move** or delete files without breaking git: + 1. To move a file that's tracked by git without breaking things, use `git mv `. + 2. To delete a file that's tracked by git, use `git rm `. + +In general it is better to commit as often as possible. Make sure what you have works and doesn't break anything else, and then commit that shit. Doesn't matter if it's a single line or a corrected punctuation mark in a comment. Commits are not precious. That way you never change too many files at a time or give others the time to do the same, reducing the need to merge files. It also makes it easier to write good commit messages :) + +### Rebasing and Why You Need It +Imagine the following situation: + - You start a branch `feature` from some commit on `master`; let's call this commit 0a. + - You make 3 commits to feature: 1b, 2b, and 3b. + - Meanwhile, 5 new commits are made on master: 1a, 2a, 3a, 4a, 5a. + - It comes time to merge feature with master. But there's a problem: + You worked from 0a, thus you can't `git merge` with master, which is at 5a. + - What to do? + +Right now your feature branch looks like this: 0a -> 1b -> 2b -> 3b. + +You need it to look like this: 0a -> **1a -> 2a -> 3a -> 4a -> 5a** -> 1b -> 2b -> 3b. + +To make this reality, you can use the *magic of rebasing*! + 1. Checkout your branch: `git checkout feature`. + 2. Hit git log. It looks like 0a -> 1b -> 2b -> 3b. + 2. *Rebase* from master: `git rebase master`. + 4. Hit git log. It now looks like 0a -> **1a -> 2a -> 3a -> 4a -> 5a** -> 1b -> 2b -> 3b. + 5. You've *changed history*. Now, if you were to go `git checkout master` and `git merge feature`, everything will work! + +You'll notice a **problem**: Because of the changed history, you can no longer push to the remote branch. This can be fixed in two ways: + * If you're an asshole: `git push --force`. You'll overwrite the entire remote branch with your local one. **CAREFUL!** + * If you can live with a merge commit: `git pull feature`. You'll create a merge commit, combining the rebased and unrebased versions, after which `git push` will work flawlessly. + +Also, a PSA: Delete your branches when they're merged!