|
@@ -1,4 +1,4 @@
|
|
|
-## [How to Mute a test](#how-to-mute)
|
|
|
+## How to Mute a test <a id="how-to-mute"></a>
|
|
|
|
|
|
- Through a PR Report
|
|
|
- Open report in PR 
|
|
@@ -16,7 +16,7 @@
|
|
|
* Set the `owner` field to the team name (see the issue for the owner's name). 
|
|
|
* Open [muted_ya.txt](https://github.com/ydb-platform/ydb/blob/main/.github/config/muted_ya.txt) in a new tab and edit it.
|
|
|
* Copy the line under `Add line to muted_ya.txt` (for example, like in the screenshot, `ydb/core/kqp/ut/query KqpStats.SysViewClientLost`) and add it to [muted_ya.txt](https://github.com/ydb-platform/ydb/blob/main/.github/config/muted_ya.txt).
|
|
|
-* Edit the branch for merging, for example, replace `{username}-patch-1` with `{username}/mute`.
|
|
|
+* Edit the branch for merging, for example, replace `{username}-patch-1` with `mute/{username}`.
|
|
|
* Create a PR - copy the PR name from the issue name.
|
|
|
* Copy the issue description to the PR, keep the line `Not for changelog (changelog entry is not required)`.
|
|
|
* Take "OK" from member of test owner team in PR
|
|
@@ -25,7 +25,7 @@
|
|
|
* Inform test owner team about new mutes - dm or in public chat (with mention of maintainer of team)
|
|
|
* You are awesome!
|
|
|
|
|
|
-## [How to UnMute a test](#how-to-unmute)
|
|
|
+## How to UnMute a test <a id="how-to-unmute"></a>
|
|
|
--IN PROGRESS--
|
|
|
* Open [muted_ya.txt](https://github.com/ydb-platform/ydb/blob/main/.github/config/muted_ya.txt)
|
|
|
* Press "Edit file" and delete line of test
|
|
@@ -33,10 +33,23 @@
|
|
|
* Edit PR name like "UnMute {testname}"
|
|
|
* Take "OK" from member of test owner team in PR
|
|
|
* Merge
|
|
|
-* If test have an issue in [Mute and Un-mute](https://github.com/orgs/ydb-platform/projects/45/views/6?visibleFields=%5B%22Title%22%2C%22Assignees%22%2C%22Status%22%2C126637100%5D) in status "Muted" - Move it to "Unmuted"
|
|
|
+* If test have an issue in [Mute and Un-mute](https://github.com/orgs/ydb-platform/projects/45/views/6?visibleFields=%5B%22Title%22%2C%22Assignees%22%2C%22Status%22%2C126637100%5D) in status `Muted` - Move it to `Unmuted`
|
|
|
* Link Issue and Pr (field "Development" in issue and PR)
|
|
|
+* Move issue to status `Unmuted`
|
|
|
* You are awesome!
|
|
|
|
|
|
+## How to manage muted tests by team <a id="how-to-manage"></a>
|
|
|
+--IN PROGRESS--
|
|
|
+### Explore your tests stability
|
|
|
+ >If you want to get more info about stability of your test visit [dashboard](https://datalens.yandex/4un3zdm0zcnyr?tab=wED) (fill field `owner`=`{your_team_name}`)
|
|
|
+
|
|
|
+
|
|
|
+### Find your muted tests
|
|
|
+ >Not all muted tests have issue in github project about this , we working on it
|
|
|
+* Open project [Mute and Un-mute](https://github.com/orgs/ydb-platform/projects/45/views/6?visibleFields=%5B%22Title%22%2C%22Assignees%22%2C%22Status%22%2C126637100%5D)
|
|
|
+* click in label with name of your team, example [link to qp](https://github.com/orgs/ydb-platform/projects/45/views/6?filterQuery=owner%3Aqp) muted tests (cgi `?filterQuery=owner%3Aqp`)
|
|
|
+* Open `Mute {testname}` issue
|
|
|
+* Perform [How to unmute](#how-to-unmute)
|
|
|
|
|
|
## Flaky Tests
|
|
|
|
|
@@ -47,14 +60,14 @@ The CI duty engineer (in progress) checks flaky tests once a day (only working d
|
|
|
- Open the [Flaky](https://datalens.yandex/4un3zdm0zcnyr) dashboard.
|
|
|
- Perform the sections **[Mute Flaky Test](#mute-flaky)** and **[Test Flaps More - Need to Unmute](#unmute-flaky)** once a day or ondemand
|
|
|
|
|
|
-### [Mute Flaky Tests](#mute-flaky)
|
|
|
+### Mute Flaky Tests <a id="mute-flaky"></a>
|
|
|
|
|
|
Open the [Flaky](https://datalens.yandex/4un3zdm0zcnyr) dashboard.
|
|
|
|
|
|
- Select today's date.
|
|
|
- Look at the tests in the Mute candidate table.
|
|
|
|
|
|
-
|
|
|
+
|
|
|
|
|
|
- Select today's date in the `date_window`.
|
|
|
- Select `days_ago_window = 5` (how many days back from the selected day to calculate statistics). Currently, there are calculations for 1 day and 5 days ago.
|
|
@@ -66,7 +79,7 @@ Open the [Flaky](https://datalens.yandex/4un3zdm0zcnyr) dashboard.
|
|
|
- Perform steps from [How to mute](#how-to-mute)
|
|
|
- You are awesome!
|
|
|
|
|
|
-### [Test is no longer flaky - Time to Unmute](#unmute-flaky)
|
|
|
+### Test is no longer flaky - Time to Unmute <a id="unmute-flaky"></a>
|
|
|
|
|
|
- Open the [Flaky](https://datalens.yandex/4un3zdm0zcnyr) dashboard.
|
|
|
- Look at the tests in the UNMute candidate table.
|
|
@@ -76,4 +89,3 @@ Open the [Flaky](https://datalens.yandex/4un3zdm0zcnyr) dashboard.
|
|
|
- If the `summary:` column shows `mute <= 3` and `success rate >= 98%` - **it's time to enable the test**.
|
|
|
- Perform steps from [How to Unmute](#how-to-unmute)
|
|
|
- You are awesome!
|
|
|
-
|