Browse Source

fix(pytest): Adding workaround to avoid systemic CI errors (#85996)

We’ve observed systemic failures in CI, causing the entire test suite to
fail due to access being denied. This issue arises from our test
fixture, which ensures no file descriptors are leaked between tests. The
problem occurs when a file descriptor is found that we are unable to
access, triggering a `PermissionError`.
The underlying cause stems from the logic used by `psutil`, which
performs more checks than necessary. Specifically, it attempts to verify
whether the path is a valid file using `os.stat()`, leading to the
access issues, as shown in the error below:
```
______ ERROR at setup of QuantizeTimeTest.test_quantizes_with_rounding_up ______
.venv/lib/python3.13/site-packages/psutil/_pslinux.py:1643: in wrapper
    return fun(self, *args, **kwargs)
.venv/lib/python3.13/site-packages/psutil/_pslinux.py:2227: in open_files
    if path.startswith('/') and isfile_strict(path):
.venv/lib/python3.13/site-packages/psutil/_common.py:526: in isfile_strict
    st = os.stat(path)
E   PermissionError: [Errno 13] Permission denied: '/root/.dotnet/corefx/cryptography/x509stores/my/226A849E790F34CD326D337D1FB66D8420F6C32B.pfx'
```
To mitigate this issue, we’ve simplified the implementation for Linux by
using a more direct approach to retrieve open file descriptors via the
`/proc` file system, which avoids triggering `os.stat()` and the
associated permission error. Since the issue appears to be isolated to
CI (which runs on Linux), we’ve limited this fix to Linux.
Ian Woodard 1 week ago
parent
commit
d6caf87fb5
1 changed files with 25 additions and 2 deletions
  1. 25 2
      tests/conftest.py

+ 25 - 2
tests/conftest.py

@@ -1,3 +1,5 @@
+import os
+import sys
 from collections.abc import MutableMapping
 
 import psutil
@@ -25,11 +27,32 @@ pytest_plugins = ["sentry.testutils.pytest"]
 # https://github.com/pytest-dev/pytest/blob/master/src/_pytest/terminal.py
 
 
+if sys.platform == "linux":
+
+    def _open_files() -> frozenset[str]:
+        ret = []
+        pid = os.getpid()
+        for fd in os.listdir(f"/proc/{pid}/fd"):
+            try:
+                path = os.readlink(f"/proc/{pid}/fd/{fd}")
+            except FileNotFoundError:
+                continue
+            else:
+                if os.path.exists(path):
+                    ret.append(path)
+        return frozenset(ret)
+
+else:
+
+    def _open_files() -> frozenset[str]:
+        return frozenset(f.path for f in psutil.Process().open_files())
+
+
 @pytest.fixture(autouse=True)
 def unclosed_files():
-    fds = frozenset(psutil.Process().open_files())
+    fds = _open_files()
     yield
-    assert frozenset(psutil.Process().open_files()) == fds
+    assert _open_files() == fds
 
 
 @pytest.fixture(autouse=True)