From 5e06e27b658d1bb2dc2427982029f42854da87a3 Mon Sep 17 00:00:00 2001 From: Waylon Walker Date: Sun, 9 Aug 2020 14:41:39 -0500 Subject: [PATCH] update for TestProject Docs --- docs/README.md | 82 ++++++++++++++++++++++---------------------------- 1 file changed, 36 insertions(+), 46 deletions(-) diff --git a/docs/README.md b/docs/README.md index 3ed4a57..bc255ba 100644 --- a/docs/README.md +++ b/docs/README.md @@ -1,41 +1,23 @@ +# Testing Client Side Scripts with Pytest in GitHub Actions +_by [waylon walker](https://waylonwalker.com)_ +This article will cover testing client side interactions with TestProject, pytest and run it inside of GitHub actions. If your GitHub repo is public this will be 100% free. This is great for learning TestProject, running integration testing on your side projects. If you need the security of running your actions from a private repo GitHub offers a very generous amount of free minutes https://github.com/features/actions#pricing-details. -As I continue to build out [waylonwalker.com](https://waylonwalker.com/) I sometimes run into some errors that are not caught because I do not have good testing implemented. I want to explore some integration testing options using GitHub's actions. - -Running integration tests will not prevent bugs from happening completely, but it will allow me to quickly spot them and rollback. - - ---- - -## 🤔 What to test first? - -The very first thing that comes to my mind is anything that is loaded or ran client-side. Two things quickly came to mind here. I run gatsby so most of my content is statically rendered, and it yells at me if something isn't as expected. For performance reasons I lazy load cards on my blogroll, loading all of the header images gets heavy and kills lighthouse (if anyone actually cares). I am also loading some information from the top open-source libraries that I have created. To prevent the need to rebuild the whole site to get the latest information I am just using the GitHub API client-side. - - -things I was looking for from features to test - -* client-side interactions -* external API - -features on my blog to consider testing - -* lazy-loaded blog cards -* GitHub Repos - - -## Repo Cards +## GitHub Repo Cards I chose to start with the GitHub repos as they seemed a bit more straight forward, and it's been a while since I have done any selenium. +[waylonwalker.com](https://waylonwalker.com/) has his favorite repositories pinned to the top of the website. The information for these cards is dynamic and pulled in from the github api client side. This means that as the pages loads JavaScript will execute to pull information from GitHubs api, then transform that data into the DOM, and render it on the page. +

Open Source cards as they look on waylonwalker.com

> here is what the GitHub repo cards look like -## TestProject.io +## Get Your Keys -I am trying out [TestProject.io](https://TestProject.io) for the first time on this project. My experience so far has been top-notch. There was an existing suite of docker images/files set up to run the TestProject agent in a docker container alongside headless chrome and firefox drivers. The first thing that you are going to need is a [TP\_DEV\_TOKEN ](https://app.TestProject.io/#/integrations/sdk) and [TP\_API\_KEY](https://app.TestProject.io/#/integrations/api). These will give TestProject access to your account so that it can automatically post results to your [dashboard](https://app.TestProject.io/#/reports) +The first thing that you are going to need is a [TP\_DEV\_TOKEN ](https://app.TestProject.io/#/integrations/sdk) and [TP\_API\_KEY](https://app.TestProject.io/#/integrations/api). These will give TestProject access to your account so that it can automatically post results to your [dashboard](https://app.TestProject.io/#/reports). * [TP\_DEV\_TOKEN ](https://app.TestProject.io/#/integrations/sdk) * [TP\_API\_KEY](https://app.TestProject.io/#/integrations/api) @@ -49,27 +31,31 @@ In your GitHub repo go to `settings>Secrets`, or append `settings/secrets` to th Secrets panel in the GitHub Repo

+--- ## Setup Dev +_optional_ -To expedite development I went ahead and set up development environment that I could log into on Digital Ocean. This allowed me to get all of my tests working a bit quicker than just running them through GitHub, but being as similar as possible. This allowed me to learn the ins and outs of setting up TestProject without needing to do a full install every time through Github's actions. +To expedite development I went ahead and set up development environment in Digital Ocean. This step is optional, you can run everything from your local machine, or completely from GitHub actions. I felt like setting up an Ubuntu Droplet in Digital Ocean gave me a very close to production feel that I could quickly iterate on. This allowed me to get all of my tests working a bit quicker than just running them through GitHub, but being as similar as possible. This allowed me to learn the ins and outs of setting up TestProject without needing to do a full install every time GitHub actions ran. [![Test Project Dev Machine setup notes card](https://waylonwalker.com/new-machine-tpio-rm.png)](https://waylonwalker.com/notes/new-machine-tpio) > I am not going to go into full dev machine setup here, but you can read my [setup notes](https://waylonwalker.com/notes/new-machine-tpio). +--- + ## 🐍 Pytest _you can see all of the tests ran with pytest on [github](https://github.com/waylonwalker/waylonwalker-com-tests/tree/master/tests)_ -I chose to go down the route of using pytest. I really liked the idea of utilizing fixtures, automatically running my test functions, and utilizing a bit of the pytest reporting capabilities. +I chose to go down the route of using pytest. I really liked the idea of utilizing fixtures, automatically running my test functions, and utilizing a bit of the pytest reporting capabilities as I was developing. TestProject does not need to run through a test framework like pytest, -**NOTE** per pytest standard practice I named the directory containing tests `tests`. While this works, TestProject.io uses this director as the default name for the project. If I were to go back I would either rename the directory to what I want to show up on TestProject.io or configure the project name inside of the config. +**NOTE** per pytest standard practice I named the directory containing tests `tests`. While this works, TestProject.io uses this directory as the default name for the project. If I were to go back I would either rename the directory to what I want to show up on TestProject.io or configure the project name inside of the config. ## conftest.py _You can see the [conftest.py](https://github.com/WaylonWalker/waylonwalker-com-tests/blob/master/tests/conftest.py) live on GitHub._ -pytest automatically imports [conftest.py](https://github.com/WaylonWalker/waylonwalker-com-tests/blob/master/tests/conftest.py) modules from the same directory that you are working from. It's common to place fixtures used across multiple files here. I placed a driver fixture in this module so that as I create more tests it will be available everywhere by default. +[conftest.py](https://github.com/WaylonWalker/waylonwalker-com-tests/blob/master/tests/conftest.py)is a common to place fixtures that are used by multiple modules. Pytest automatically imports all [conftest.py](https://github.com/WaylonWalker/waylonwalker-com-tests/blob/master/tests/conftest.py) modules from the same directory that you are working from. This is a great place to include TesProject driver fixtures. Note that when you use a fixture as an argument in another function, the fixture will do setup, pass anything from the yield statement to the test function, run the test function, then run teardown. > conftest.py stores fixtures for all modules in a directory. @@ -88,19 +74,22 @@ def driver(): yield driver driver.quit() ``` -> Look at the full version of [conftest.py](https://github.com/WaylonWalker/waylonwalker-com-tests/blob/master/tests/conftest.py) -The above sample is a bit **simplified**. I ran into some inconsistencies in the real version and found that some tests had a better pass rate if I added a wait. I ended up with a `driver` and a `slow_driver` fixture. +> The full version of [conftest.py](https://github.com/WaylonWalker/waylonwalker-com-tests/blob/master/tests/conftest.py) is available on GitHub + +The above sample is a bit **simplified**. I ran into some inconsistencies in the real version and found that some tests had a better pass rate if I added a `time.sleep` statement. In my full project I ended up with a `driver` and a `slow_driver` fixture, this allowed me to have a driver that waited for JavaScript to execute just a bit longer. ## test_repos.py -_see the full [testrepos.py](waylonwalker.com/testrepos.py) on GitHub_ +_The full version of [testrepos.py](waylonwalker.com/testrepos.py) is available on GitHub_ -I have initially set up 3 different tests for the repo cards. I set a list of repos that I expect to show up in the cards. These tests are quite easy to do with TestProject.io as it is using selenium and a headless browser to execute javascript under the hood. +I have initially set up 3 different tests for the repo cards. I set a list of repos that I expect to show up in the cards. These tests are quite easy to do with TestProject.io as it is using selenium and a headless browser to execute javascript under the hood. The `REPOS` area created as a global list here, and can easily be refactored into a config file if the time ever comes that they need to be. If you are not familiar a **headless browser** runs the engine as your browser without a graphical user interface. JavaScript gets fully loaded and parsed, and the dom is completely interactive programmatically. +Read through the docstrings of each function as it describes what happens at each step. + ``` python """ Test that GitHub repo data dynamically loads the client-side. @@ -158,20 +147,22 @@ def test_repo_stars_loaded(slow_driver): ## Forum _[forum.TestProject.io](https://forum.TestProject.io/t/install-agent-inside-github-actions/2334/3)_ -Before jumping into the real action. I quickly wanted to mention the **amazing** ✨ discord server that they have going. - -I was a bit confused about how to set up TestProject.io inside of actions. I was with a prompt response linking me to the exact example I needed. The tests were written in java, but they had set up the docker-compose steps that I needed. +I was a bit confused about how to set up TestProject.io inside of actions. I was welcomed with a prompt response linking me to the exact example I needed. The tests were written in java, but they had set up the docker-compose steps that I needed. --- ## GitHub Actions 🎬 -_[test-waylonwalker-com.yml](https://github.com/WaylonWalker/waylonwalker-com-tests/blob/master/.github/workflows/test-waylonwalker-com.yml)_ +_[test-waylonwalker-com.yml](https://github.com/WaylonWalker/waylonwalker-com-tests/blob/master/.github/workflows/test-waylonwalker-com.yml_ Now that I have my GitHub repo setup with my [tests](https://github.com/WaylonWalker/waylonwalker-com-tests/tree/master/tests) successfully running in pytest, let's get it running inside of GitHub actions automatically. +Actions are GitHub's solution to CI/CD. It allows you to execute code within a virtual machine managed by GitHub that can get extra information from your repo such as secrets, which we set up at the beginning. What gets ran, how it gets ran, and when it gets ran are all configured inside of a `yaml` file. + + ``` yaml +# .github/workflows/test-waylonwalker-com.yml name: Test WaylonWalker.com # Controls when the action will run. Triggers the workflow on push or pull request @@ -214,7 +205,7 @@ jobs: TP_AGENT_URL: http://localhost:8585 ``` -In the test job you can see that we have rendered the [TP\_API\_KEY](https://app.TestProject.io/#/integrations/api) into the [docker-compose.yml](https://github.com/WaylonWalker/waylonwalker-com-tests/blob/master/.github/ci/docker-compose.yml) using `envsubst` file so that TestProject has access to it. We have also exposed our [TP\_DEV\_TOKEN ](https://app.TestProject.io/#/integrations/sdk) to pytest. +In the test job you can see that I have decided to run the test job on `ubuntu-latest`. The first three steps are a bit of boilerplate to checkout the repo, setup python 3.8, and pip install our `requirements.txt`. Next the [TP\_API\_KEY](https://app.TestProject.io/#/integrations/api) as been rendered into the [docker-compose.yml](https://github.com/WaylonWalker/waylonwalker-com-tests/blob/master/.github/ci/docker-compose.yml) using `envsubst`, `docker-compose` has been started, and waited for the agent to be ready. I have also exposed our [TP\_DEV\_TOKEN ](https://app.TestProject.io/#/integrations/sdk) to pytest and ran pytest. ## docker-compose.yml @@ -260,7 +251,7 @@ services: ## ⌚ Waiting for the Agent to register _[wait for agent.sh](https://waylonwalker.com/waitforagent.sh)_ -I think the most interesting part of the workflow above is how we wait for the agent to register. The shell script is a bit terse, but it looks for exceeding the `max_attempts` allowed or that the agent has started by using its `/api/status` rest API. This prevents us from wasting too much time by setting a big wait, or trying to move on too early and running pytest without a running agent. +I think the most interesting part of the workflow above is how we wait for the agent to register. The shell script is a bit terse. It looks for exceeding the `max_attempts` allowed or that the agent has started by using its `/api/status` rest API. This prevents us from wasting too much time by setting a big wait, or trying to move on too early and running pytest without a running agent. ``` bash trap 'kill $(jobs -p)' EXIT @@ -282,7 +273,8 @@ done ## TestProject.io Dashboard 〽 -One one of the coolest features that you get from TestProject.io are the [reports](https://app.testproject.io/#/reports) dashboard. To me, this felt like a premium feature for **free**. Here you can see a time-series plot of your tests success rate over time. It gives you a bit of an ability to slice in, but not a lot. Some of the filters are pre-canned, like the past 2 days are past 30 days cannot be customized. +Once the tests are running they will appear in the TestProject dashboard. There are a number of failures that happened early on in development of the tests, but once they started passing they continued to pass. +

@@ -305,10 +297,8 @@ As you drill in you can see individual tests that have been run, select them, an />

+## Authored by Waylon Walker -## Overall 😄 +This page was created by [waylon walker](https://waylonwalker.com) based on his article ([Integration testing with Python, TestProject.io, and GitHub Actions](https://waylonwalker.com/blog/testproject-io-py-actions/)). -The experience I had setting up TestProject.io to run inside GitHub's actions was great. It was fairly simple to set up and get running with many of the greatest integration testing tools of today, selenium, chrome, firefox. - -Now I am going to turn the test frequency down a bit.