Using Github with docassemble

Ways to use GitHub in combination with docassemble. GitHub itself has decent documentation for its own features.

Why?#

GitHub lets you collaborate with your team more easily, helps other people help you if you run into problems, saves working code so you can come back to it if new edits break something (a savepoint), and lets others find your code if they want to do something similar.

These are some thing GitHub lets you do and why you might want them.

Why: branches and commits#

  • Avoid introducing breaking changes to the working code.
  • Work on multiple features/changes at the same time, keeping each isolated from the others. Either collaborators can work that way or one individual and isolate specific issues to work on.
  • Try implementing a feature multiple different ways.
  • Try out and review each others' code without messing up your own code.
  • Clearly see what has changed between versions of your code when you lose track.
  • Get reviews/tests from other people to make sure code is working before adding it to the main/master working branch.

Why: issues#

  • Track tasks that need doing and add context to them.
  • Track decisions that have been made.
  • Search the tasks when looking for relevant information.
  • Assign tasks to specific people.
  • See the tasks that were assigned to you specifically.

Why: community#

  • People with more experience can pull your code in more easily to experiment with it and troubleshoot more effectively.
  • When your time on the project is over, other people can take up the code to continue developing it.

Setup#

Get a docassemble account#

  1. Find server administrator.
  2. Give them your email.
  3. Ask them to invite you to the server with developer privileges.
  4. When you get an invitation email, follow the link to the server.
  5. Make sure you can sign in.

Sign in

Join GitHub#

  1. Make sure you have a GitHub account
  2. If this is a new GitHub account, adjust settings to keep your email address private and make sure your email does not get stored when you send your code to GitHub.

Join your organization#

  1. Make sure your organization invites you to their GitHub Organization or invites you to their Team.
  2. Follow the link in your email to accept the invitation.

Connect GitHub to your docassemble account#

Summary

  1. Go to your docassemble "Profile" > "Other Settings" > "GitHub integration"
  2. Authorize docassemble to access your GitHub account

Details

Open docassemble and select "Profile" from the drop down menu.

Open your profile from the sign-in dropdown menu in the top right

Tap "Other Settings" and then "GitHub integration" from the drop down menu.

Navigate to GitHub the integrations page

Tap "Configure".

Tap 'Configure' on the GitHub integrations page

Tap to authorize your docassemble server on GitHub. It will restart the server.

Authorize docassemble

Once the server is done you can return to the Playground.

Make issues#

Keywords: Tasks, todo lists, bugs, features, documentation, milestones

When to make an issue#

You can make an issue to:

  1. Document a decision. It is useful to include who was involved in the discussion and why that decision was made. Some example decisions might be about language choices, question-type, order, and flow. When new people come to work on the form we don't want to duplicate work that's already been done or to re-interrogate old decisions without the context of why they were made.
  2. Add a new task. For example, problems that needs to be researched or programmed. Research could involve consulting a specific SME (subject matter expert) and you, or someone commenting on the issue, could note their name.
  3. Categorize and organize tasks. For example, MVP items, version 2 improvements, and bugs.

Give all the links possible! All context!

What to do with issues#

See the GitHub docs about:

  1. Making an issue.
  2. Assigning the task to yourself or another member.
  3. Adding labels.
  4. Creating milestones.
  5. Adding a milestone to an issue (you can only have one milestone one per issue).

New forms#

Prepare a new Project#

You can also see the docassemble docs on how to create new Projects.

caution

Follow the Project naming rules or you will get an error: a unique name that starts with an alphabetical letter and uses only alphanumeric characters.

Summary

  1. Go to "Playground" > "Manage Projects", create a new Project, and name it appropriately.
  2. Go back to "Playground" and upload your files or make a new file. The name of the YAML file should be the name of your form.
  3. Go to Packages and save your package. Its name should be based on the form name and follow conventions.
  4. Commit/push the package to GitHub.

Details

In the Playground, tap the top left dropdown and select "Manage Projects".

Navigate to "Manage Projects"

Make a new Project and name it appropriately.

Create a new Project

Name a new Project

Tap your new Project's name.

Open a Project

The Playground will open the Project with a default "test.yml" file. Either add a new file or upload your generated .yml file by tapping "Upload".

Upload a YAML file in the Playground

Create your Package and push it to GitHub.

Create a new YAML file#

Summary

Go to the Playground, tap "Add", and give the file an appropriate name.

Details

Go to the Playground and tap "Add".

Tap "Add"

Give the file an appropriate name. Then save the file.

Name and save the YAML file

Your save options should change below the code area to indicate the save was successful.

The editor buttons change once the file has been saved the first time

Upload a YAML file#

See the docassemble docs about Playground YAML files.

Upload a package#

See the docassemble docs to use a zip file to upload a package.

Template Files: Upload the PDF or DOCX file that your code fills out#

See the docassemble docs about uploading template files.

Summary

Navigate to Playground > "Folders" > "Templates" and upload your file.

Details

Go to the Playground and make sure you're in the right Project.

Check your Project's name

Tap "Folders" and then "Templates".

Open the Templates folder

Your file will be in the list below of available templates.

Open the Templates folder

Create a new repository#

aka. Create a new Package, commit or upload a new Package to GitHub for the first time.

This is about creating a brand new Package from new files. If you are creating a new Package from the files of an old Package, we hope to develop a section for that in the future.

A docassemble Package bundles code, metadata, dependencies, and your PDF's or DOCX files into one place for you and others to access. You get to pick and choose the files that you want to go into that particular Package.

Also see docassemble docs about pushing a Package to GitHub by committing and about Packages in general.

caution

Sometimes docassemble will not create a new repository on GitHub. If your Package name matches the name of a Package that is already in your GitHub repository or in a repository of an organization you belong to, docassemble will try to push to that repository instead of creating a new repository.

In most cases, you will get a "merge conflict" error when this happens.

Summary

  1. Go to "Packages".
  2. Give your new Package a name.
  3. Save it.
  4. Commit to GitHub.

Details

Switch to the correct Project if you need to. Go to the Project's Packages folder. Name your Package based on your document file name. Example: The name of the marriage_without_delay.pdf package would be "MarriageWithoutDelay".

Name the Package

Scroll down to the boxes further down the page. Use cmd/ctrl + click to select the YAML files and documents you want to have in your repository. For example, here we would select plaintiff_s_motion_to_modify.yml and plaintiff_s_motion_to_modify.pdf.

Select your files

Scroll down to the bottom and tap Save.

Save the Package settings

If you have configured your GitHub account, once your package saves and the page reloads the GitHub button will appear near the Save button. Tap GitHub.

Tap the GitHub button

You must write a message describing what this commit is for. Read more about commit messages here. This is the first commit message, though, and a common first commit message is "Initial commit". When your message is ready, tap "Commit".

Write a commit message

Look at the bottom of the Package screen again. Just above the Github button will be links to your new repository.

Links to the repository

You can also read the docassemble docs on pushing your package to GitHub by committing

Transfer ownership to your organization#

After the repository's ownership is transferred, only people who have permissions in your organization will be able to work on the code. Your organization can give people permissions through GitHub.

Read the GitHub docs on transferring ownership of your repository.

If your organization has teams, pick which teams can have access to this repository.

Transfer to teams

Check your organization's GitHub page to make sure your project appears there. You may have to ask give the administrator the new link to the repository and ask them to give the rest of the team permissions on it.

caution

If you get a pink error message that says you don't have permissions, you have not been added as a member of that GitHub organization.

Cannot transfer repo

Sharing code#

Mistakes will happen. You will accidentally include changes you didn't mean to include or push to the wrong branch. The great thing is that you can't break things permanently. GitHub uses git for version control and it's specifically made for rolling back changes. Docassemble does not give full access to its features and its a complex ecosystem, so someone more experienced may need to lend a hand, but it's all there. Your gut won't believe it right now, but you will come to trust the process.

Commit - EARLY AND OFTEN#

aka. Savepoints, saving your progress, bookmark where your code is at right now.

Commits help preserve your work, help you find old working code when something breaks, and help you track down bugs by peeling back only a few changes at a time till you get to a working version.

Note: When you first start, it's hard to remember to stop and commit. Create a daily alarm in your phone to commit all code that you haven't yet committed that day.

COMMIT EARLY AND OFTEN (bears repeating). GitHub can't help you if you don't use it.

You can commit to a branch as many, or as few, times as you want. Developers often wait to commit code until they have it working, but they might commit broken code to make it easier for other people to help them or to save progress. If you are concerned about ruining working code, you can make a new branch for the broken code and commit to that branch.

  1. Go to the correct Project and make at least one change (edit or add a file).
  2. Go to "Packages".
  3. In the branches dropdown, select the branch you want to add your changes to or create a new branch.
  4. Write a commit message (required)
  5. Tap 'Commit'
  6. See your commit on GitHub!

Making a new branch for a new goal or bug fix#

Make a new branch when you first start working on a feature or goal. For example, income questions or a bug fix. It preserves your previous work so that it isn't affected by these changes and lets other people see the changes before they're completely added in. This is very similar to committing changes, but has a couple additional steps.

  1. Make at least one change. Edit code or include a new file with cmd/ctrl + click.
  2. Go to "Packages" and tap "GitHub".
  3. In the branches dropdown, select New Branch.
  4. Give the branch a snake_case name.
  5. Commit your changes with a commit message.

You will now be able to see your new branch on GitHub! If you want, you can compare your branch to any other branch and see the exact differences.

Branch name

A branch name reminds you and your collaborators, at a glance, what topic, problem, or feature the branch is for. 1 to 3 words separated by underscores is usually a good guideline. Example: income_questions or income_calculations

More advanced: Include the issue number and type of goal of the branch - a feature, a fix, etc. Example: 116_fix_income_calculations or 42_feature_deductibles.

Branches vs. commits#

Branches should encapsulate specific goals, like adding a set of questions or creating automated tests for one path in your interview.

Commits should encapsulate one particular action in a branch. One commit might be adding one question or getting one part of the code working, even if other parts still need some work. Together, individual commits add up to the complete feature. A commit might fix typos, improve readability of the first three questions, or add the question about debt.

Request others test your code#

aka. Getting a review, making a PR, a pull request, a merge request.

You can see a list of your repository's branches in GitHub and you can make a pull request from there to merge a branch's changes into your default branch.

  1. Create a pull request.
  2. If this PR is related to an issue, edit the description to link to the issue. You can use the same format as described in the section about closing an issue with a PR.
  3. Edit the description to add a checklist of things the reviewer needs to check or test.
  4. Request a review from another member.
  5. Send a message to that person to let them know you've assigned them as a reviewer.

Merge conflicts#

aka. Errors combining two branches, trouble merging, trouble combining, resolve merge conflicts, fixing merge conflicts between two branches, branches diverged

Sometimes when you start to make a pull request, GitHub will tell you that you have a merge conflict. This means that while you were making changes to your branch, changes were also made to the branch you are trying to merge with and they changed the same lines of code. Git will not decide which bits of code to keep where - you have to decide that.

These are unavoidable and a pain to deal with. Everyone runs into them now and again and they are tricky to resolve without introducing bugs.

tip

Ways to reduce the likelyhood of merge conflicts:

  1. If possible, make the goals of your branches small so that you only make a few changes on a branch before making a pull request with it.
  2. Merge pull requests in the order they were made.
  3. Avoid editing the same sections of code at the same time.
  4. If you are working with collaborators, try to make sure you are working on tasks that do not edit the same lines.

You can still make your pull request, but GitHub will prohibit merging. There are two ways GitHub might handle merge conflicts:

  1. GitHub allows you to 'resolve' the conflict in GitHub itself by editing the files in GitHub.
  2. GitHub is unable to help you resolve the issue (it doesn't offer the option).

Either way, the most reliable way to deal with this situation is, unfortunately, to redo your changes on top of the new code:

  1. Make the pull request as usual, but simply use it as a tool. The changes you made will be highlighted in the file comparison tab.
  2. In docassemble create a new Project and pull in the other branch (the one you want to merge with).
  3. Using the file comparison as a guide, manually add your changes back.
  4. Make sure functionality affected by your code changes still works the way it should.

Errors when committing from docassemble#

aka. I made edits, but docassemble says I can't push to GitHub.

When you get an error committing (and pushing) from docassemble, nothing has been changed on GitHub. One of three things is probably going on.

  1. You have technically not made any changes to the code that is actually being pushed to GitHub.
  2. You lack permissions for that repository.
  3. The changes between your branch and the branch on GitHub do not match up. That is, there's a merge conflict.

Scroll to the bottom of the page of the error. The text on this page is a step-by-step log of what is happening in git, the version control technology GitHub uses, and the most recent message is at the bottom. Ignore the line talking about a Detached head - this is a red herring. It's actually a natural intermediate part of the process.

nothing to commit, working tree clean#

This means git cannot detect any changes in your code. You cannot commit anything from docassemble if you have not made changes, not even to a new branch. These are possible ways to confirm the problem:

  1. If you created a new file you may have not added it to your Package yet. In the image below, only court_form.docx has been selected.

Unselected template files

On a Mac use cmd + click to select a file you need. On a PC use ctrl + click. If there are two files with very similar names, make sure the name of the file you need exactly matches up with the name you are clicking.

  1. You may have forgotten to save the changes you made to a file. If you still have the file open, you can tap 'Save'. If the file is closed, you will have to make the changes again.
  2. You may not have made any changes to files even if you thought you did. To sanity check changes to a code file:
    1. Find a diff checker, like https://www.diffchecker.com/.
    2. Go to the Playground to the file you think you've changed.
    3. Copy all the text there.
    4. Go back to the diff checker and paste it in the right-hand column.
    5. In GitHub, navigate to the file you think you changed.
    6. Copy the text in it.
    7. Put it in the left side of the diff checker.
    8. Press the button to compare the text.

If you do not see differences between the files, you have not made the edits you think you have. Are you in the right Project? Did you change a file with a very similar name to another file?

403#

A 403 error, remote: Permission to ... denied to ..., means you have not been given permission to change things in that repository. The person or organization that owns the package will have to give you or the team you are on 'write' permissions. They could also add you to a team that already has write permissions.

If you are trying to make a new repository on your GitHub account, you may belong to an organization that owns a Package by the same name. You may not have permission to change that repository. You will either have to change the name of your Package, have the organization delete that repository, or have the admin give you or your team 'write' permissions as described above.

CONFLICT (content)#

The text CONFLICT (content): Merge conflict means the branch you are trying to push to changed while you were making your changes. It is what a merge conflict looks like in docassemble in git. Simply make a new branch and commit the changes there. Work out the merge conflicts afterwards.

Testing someone else's code#

Reviewing PRs#

When someone assigns you as a reviewer for a pull request, you will be able to see it in your GitHub pull request list or your notifications page. You can read more in [the GitHub documentation on seeing your issues and pull requests](see how to navigate therehttps://docs.github.com/en/github/managing-your-work-on-github/viewing-all-of-your-issues-and-pull-requests). To just look at the pull requests for a repository, go to the 'Pull Requests' tab on the repository's home page.

Don't let pull requests hang around too long. Other code will get edited and merge conflicts will crop up, making it hard to bring the code back together.

The requester should have left notes on what needs to be tested in the description of the pull request (PR).

Summary

  1. Pull the code into the a new Project in your Playground.
  2. If the behavior is as expected, approve the changes and merge the branch.
  3. If the behavior is not as expected, request changes.

Details

Make a new Project in docassemble and pull the code from the branch that's making the PR. You can see the name of the branch on GitHub in the pull request itself:

Name of pull request branch

Test the behavior that is affected by the changes in their code. Sometimes it can help to look at the pull request (PR) file comparison tab in GitHub to see what code has been changed. If it works as expected:

  1. Write a review saying what you tested and the results.
  2. As described in that documentation, select 'Approve'.
  3. Merge their branch.
  4. Depending on your agreed upon practices, delete the branch.

If the interview does not work as expected:

  1. Write a review that describes what you saw that you think needs changing.
  2. As described in that documentation, select 'Request Changes'.
  3. Let the requester know about your review.

Finally, delete the Project you created for the review.

note
  1. You can make comments connected to specific lines of code.
  2. You can make a pull request with someone's branch even if the person is not there as long as you have write permissions on the repository.

Get GitHub code into your Playground#

aka. Uploading your code to the Playground, uploading someone else's code to the Playground, reviewing, pulling, getting your old code back, seeing someone else's code, working on someone else's code

warning

WARNING: All the files in the repository will replace files of the same name in this Project. Imagine being on your computer, copying a bunch of files to a folder, and getting the message "This file already exists in this folder. Do you want to replace it?". It's the same thing, but docassemble doesn't give you that warning.

If you want to save your changes, create a new branch for it before pulling the new code in.

Summary

  1. Make a new Project in docassemble and go to "Packages".
  2. In the menu of blue text at the top, tap 'Pull'.
  3. Go to GitHub and get the url (web address) of the repository that has the code.
  4. From the dropdown, pick the branch you want to pull from and tap 'Pull'.
  5. Go back to your Playground and play!

Details

Make a new Project in docassemble then go to your Packages folder. In the menu at the top, tap "Pull".

Navigate to the Package Pull page

Copy the GitHub url of the repository you are pulling from.

Get the repository url

Make sure there is no trailing slash.
Yes: https://github.com/some_user/docassemble-SomeRepository
No: https://github.com/some_user/docassemble-SomeRepository/

Back in docassemble, paste the url in the text field.

Paste the repository url

Tap the dropdown menu and select the branch you want to bring in.

Choose the right branch

Tap "Pull".

Tap the Pull button

When you return to the Playground, you should see a file from the code you have pulled in.

Navigating#

Find docassemble pages like the Playground and the Packages pages.

Go to the Playground#

Go to the Playground through the dropdown

Go to the Package#

Playground > Folders > Packages

Getting to Packages from the Playground folder

Advanced/Needs bandwidth#

Commit messages#

A commit message is required to make a commit. The most important part of a It describes the specific changes you just made that are working towards the goal of the branch. You can write a commit message as if it is an item in a todo list. It helps to start with a verb. Example: Update cash income question.

When you look back on your list of such commits, you would be able to construct a todo list of what needed to get done. If commits need to be undone (reverted), these might help you recover the tasks that need doing.

Regular commit messages can also link to or close issues, just like the commit messages of a PR can reference or close issues. Pro tip: they're actually all just regular commit messages.

Close an issue with a PR#

See the GitHub docs on closing an issue with a PR. The issue will only be closed when the PR is merged into your default branch (usually the one called 'main'). Example: Add all financial questions, fix #15

You can also just create a link to an issue with a PR or commit by simply omitting the closing keywords shown in the GitHub documentation. Just use the pound sign (#) and the number of the issue. Example: Add question about assets, #15

Which branch am I editing?#

In docassemble it can be hard to tell if you are in the correct Project or which branch's files you are editing.

  1. Go to Playground > Folders > Packages.
  2. Scroll down to the GitHub section.
  3. You will see the name of this package's branch in the links above the GitHub button and can follow a link to go to the repository.

Links to GitHub repository

Make a new package with selected files#

aka. Make a new package, cherry pick some files to make a package, select individual files.

  1. If you have not done so yet, pull in the repository branch with the files you want.
  2. Stay in the Packages page.
  3. At the top of the page, tap "Add".
  4. Give the new package a name using the appropriate rules.
  5. Select the file you want in this package. They can be the same files as before.
  6. Commit

Edit a package without write permission#

aka. forking a repository, making a PR from a forked repository, working with someone else's repository.

If you do not have write permissions on a repository, you can still see play with its code and even make a pull request to help them change the code.

  1. Make sure the repository is not owned by an organization you belong to. If it does, ask an admin to set your permissions correctly.
  2. On GitHub, fork the repository.
  3. In docassemble make a new Project and pull from that forked repository.
  4. Make a small edit and make a commit to make sure you are able to push to your fork.
  5. You can now treat this as you would treat any other package.

When you make a pull request from one branch of your fork to another branch of your fork, make sure the chosen branches in the dropdowns are correct. GitHub will automatically set your pull request to use the original owner's repository instead of your own.

If you later do want to offer your changes to the owner of the original repository, you can make a pull request from your fork.