Browse Source

feat(codecov): after_n_builds per flag & codecov comment in PRs (#46942)

Codecov now supports `after_n_builds` per flag rather than a shared
value.

This means that FE & BE changes would not create a codecov check on PRs
until the exact number of codecov checks have uploaded their results. FE
changes currently upload 5 codecov results while BE changes upload 11.

We will have to be careful to update the number of builds for each as
more CI checks are added.

This [BE PR](https://github.com/getsentry/sentry/pull/46943) (based on
this branch) created a status check:
<img width="446" alt="image"
src="https://user-images.githubusercontent.com/44410/230431632-3bb48d08-e705-4aa1-b170-89499a07350d.png">

There are some issues with [the FE
PR](https://github.com/getsentry/sentry/pull/46944) that I will figure
out in a follow up PR.

---------

Co-authored-by: Matt Gaunt-Seo @ Sentry.io <112419115+mattgauntseo-sentry@users.noreply.github.com>
Armen Zambrano G 1 year ago
parent
commit
b41da635ef
1 changed files with 7 additions and 11 deletions
  1. 7 11
      codecov.yml

+ 7 - 11
codecov.yml

@@ -1,10 +1,3 @@
-codecov:
-  notify:
-    # The minimum amount of codecov reporting checks that need to complete
-    # For FE we have 5 checks and for BE we have 11
-    # TODO: Remove this when we have support of after_n_builds per flag
-    after_n_builds: 5
-
 # Setting coverage targets per flag
 coverage:
   status:
@@ -15,16 +8,14 @@ coverage:
       frontend:
         # codecov will not fail status checks for master
         only_pulls: true
-        # This will be needed until codecov supports after_n_builds for flags
-        informational: true # Do not fail the check
+        informational: false # Fail the check
         target: 60%
         flags:
         - frontend
       backend:
         # codecov will not fail status checks for master
         only_pulls: true
-        # This will be needed until codecov supports after_n_builds for flags
-        informational: true # Do not fail the check
+        informational: false # Fail the check
         target: 90%
         flags:
           - backend
@@ -59,10 +50,15 @@ flags:
     paths:
     - "static/app/"
     carryforward: true
+    # FE uploads 5 coverage reports. This property ensures codecov waits
+    #  for all reports to be uploaded before creating a GitHub status check.
+    after_n_builds: 5
   backend:
     paths:
     - "src/sentry/**/*.py"
     carryforward: true
+    # Do not send any status checks until N coverage reports are uploaded
+    after_n_builds: 11
 
 # Read more here: https://docs.codecov.com/docs/pull-request-comments
 comment: