FS#79822 - [pwndbg] pwndbg wants a virtualenv, but no venv provided.

Attached to Project: Arch Linux
Opened by astro angelfish (astro_angelfish) - Saturday, 30 September 2023, 12:23 GMT
Last edited by Buggy McBugFace (bugbot) - Saturday, 25 November 2023, 20:20 GMT
Task Type Bug Report
Category Packages: Extra
Status Closed
Assigned To Antonio Rojas (arojas)
Levente Polyak (anthraxx)
Architecture All
Severity High
Priority Normal
Reported Version
Due in Version Undecided
Due Date Undecided
Percent Complete 100%
Votes 4
Private No


pwndbg in arch linux version is unable to launch with `Cannot find Pwndbg virtualenv directory: /usr/share/pwndbg/.venv: please re-run setup.sh`.
The upstream had already fixed it in commit https://github.com/pwndbg/pwndbg/commit/f3914e26968ca3cc0a1f06a29cd54913beb662ff with an environment variable check but had not released it yet, but is likely to release in October 2023.
The package is unlikely to be available until the next release unless invoking its `setup.sh` to create a virtual environment as a temporary fix or build directly from source.

Additional info:
* package version(s)
> 2023.07.17-1
* config and/or log files etc.
> not available
* link to upstream bug report, if any
> not available. but there are a few related issues at https://github.com/pwndbg/pwndbg/issues/1866, https://github.com/pwndbg/pwndbg/issues/1847

Steps to reproduce:
1. Do `pacman -S pwndbg`
2. Do `echo source /usr/share/pwndbg/gdbinit.py > ~/.gdbinit`
3. Do `gdb`, and pwndbg refuses to start since no virtual environment is provided.
This task depends upon

Closed by  Buggy McBugFace (bugbot)
Saturday, 25 November 2023, 20:20 GMT
Reason for closing:  Moved
Additional comments about closing:  https://gitlab.archlinux.org/archlinux/p ackaging/packages/pwndbg/issues/1
Comment by astro angelfish (astro_angelfish) - Sunday, 01 October 2023, 05:28 GMT
I've also tried to fix this issue by myself by abusing `PWNDBG_VENV_PATH=PWNDBG_PLEASE_SKIP_VENV` environment variable with pwndbg-git, copying the `pwndbg` directory under `/usr/share/pwndbg` into `/usr/lib/python3.11/site-packages` (required since no virtual environment), but got these:
Traceback (most recent call last):
File "/usr/share/pwndbg/gdbinit.py", line 76, in <module>
import pwndbg # noqa: F401
File "/usr/lib/python3.11/site-packages/pwndbg/__init__.py", line 9, in <module>
import pwndbg.commands
File "/usr/lib/python3.11/site-packages/pwndbg/commands/__init__.py", line 17, in <module>
from pwndbg.heap.ptmalloc import DebugSymsHeap
File "/usr/lib/python3.11/site-packages/pwndbg/heap/ptmalloc.py", line 19, in <module>
import pwndbg.disasm
File "/usr/lib/python3.11/site-packages/pwndbg/disasm/__init__.py", line 40, in <module>
"rv32": CS_ARCH_RISCV,
It seems to be related with old version of python-capstone (https://archlinux.org/packages/extra/x86_64/python-capstone/), which is 4.0.2 and had already been flagged out of date. The missing constant is introduced at 5.0.0.
I think a virtual environment is recommended before the python-capstone package updates.
By the way I guess pwndbg also wants gdb-pt-dump in the same directory since I saw the `gdbinit.py` registers it into path of python.

So I suggest a temporary fix is creating a virtual environment for pwndbg instead of using environment of system.
Comment by loqs (loqs) - Wednesday, 04 October 2023, 14:22 GMT
Have you tried updating capstone to 5.0.1 then rebuilding pwndbg and any other packages you have installed that depend on capstone?
python-pyocd [1] and ropper [2] appear to be the only issues with updating capstone.

[1] https://github.com/pyocd/pyOCD/issues/1598
[2] https://github.com/sashs/Ropper/blob/v1.13.8/requirements.txt#L1
Comment by astro angelfish (astro_angelfish) - Thursday, 05 October 2023, 02:48 GMT
I have successfully launched pwndbg with capstone 5.0.1, after doing `cp -r /usr/share/pwndbg/pwndbg/ /usr/lib/python3.11/site-packages/`.
But I also noticed a few packages wants legacy capstone when trying to test on real machine: `radare2` and `rizin`
At upstream of rizin it is likely to have already added support to capstone v5 in at most v0.7.0 while arch package is still at v0.6.2
Radare2 is also OK when build directly from source.
Should they be flagged out of date?
Comment by loqs (loqs) - Thursday, 05 October 2023, 03:51 GMT
Did you change -D use_capstone_version=v4 to -D use_capstone_version=v5 in the radare2 and rizin PKGBUIDs?
Comment by astro angelfish (astro_angelfish) - Thursday, 05 October 2023, 11:19 GMT
No I didn't. Instead I tested pwndbg alone in a docker environment because I don't want to mess up my disk.
After testing pwndbg I also built them from upstream repository directly in the docker and found they works fine for me, but I did not change any of their PKGBUILDs because I forgot that. So I think they might need a rebuild in arch package repository when updating.
Maybe I'd better to test them by modifying their PKGBUILDs now.
Comment by berilac (berilac) - Tuesday, 31 October 2023, 14:45 GMT
It appears that unfortunately this latest release now requires a `venv` by default, and only later did they add the `PWNDBG_PLEASE_SKIP_VENV` check (not included in this upstream release).
I've partially tried to manually set up the venv, but then the local `pip install -e <path to python lib>` fails presumably because we don't copy the `pyproject.toml` during install.

Really, if we're going to use this package, then the `PKGBUILD` needs to be updated.
As a side note, the `PKGBUILD` also seems to have a superfluous `sed` command.

Should the build actually make use of the provided `setup.sh`? I'm inclined to say no.
But then we need to handle the `venv` logic somehow?

Should we enforce a `venv` as though we had used `setup.sh`?

Or would it be sufficient to `patch` the `gdbinit.py` (until the next release I guess) to at least include the `PWNDBG_PLEASE_SKIP_VENV` check?
Then we can default to setting that and skip the `venv` entirely - or offer a choice.
Comment by loqs (loqs) - Tuesday, 31 October 2023, 18:11 GMT
@berilac if you rebuild pwndbg with the attached diff applied and using the updated python-capstone; does that resolve the issue? The patch aims to make venv optional and not require setting PWNDBG_PLEASE_SKIP_VENV. It is based on upstream commit [1].

[1]: https://github.com/pwndbg/pwndbg/commit/f3914e26968ca3cc0a1f06a29cd54913beb662ff
Comment by loqs (loqs) - Thursday, 02 November 2023, 21:18 GMT
https://drive.google.com/drive/folders/1_DBSakAzSi6gb-bbshrE9sJg27lubAXT?usp=sharing contains capstone 5 and packages rebuilt against it (Note does not include rebuilt qemu packages). Does using those packages resolve the issue?
Comment by berilac (berilac) - Saturday, 04 November 2023, 09:35 GMT
@loqs Cheers, much appreciated. Trying asap (initially the patch solution) - have been bit more afk than expected.
Will post any discoveries.
Comment by berilac (berilac) - Saturday, 04 November 2023, 14:17 GMT
@loqs The `PKGBUILD.diff` still results in a check that is dependent on `PWNDBG_PLEASE_SKIP_ENV`. Afaict.
But now getting an error: `ModuleNotFoundError: No module named 'pwndbg'`. Possibly unrelated and potentially masking whether or not the patch helps.
Have to try and solve this first.
Comment by berilac (berilac) - Saturday, 04 November 2023, 14:37 GMT
Which looks to be related to the capstone issue you've mentioned, so looking into that now.
Comment by berilac (berilac) - Saturday, 04 November 2023, 14:45 GMT
@loqs I have now tried your patched binaries, but still having the same `python` module import error.
`pacman -Qi python-capstone`, etc are showing the `v5` packages.
Comment by loqs (loqs) - Saturday, 04 November 2023, 14:56 GMT
I added pwndbg-2023.07.17-1.2-any.pkg.tar.zst which uses [1] except for the changes to requirements.txt which did not apply and setup.sh which is not used. diff of changes [2] attached. This is intended to rule out any issues introduced by my code changes.

[1]: https://github.com/pwndbg/pwndbg/commit/f3914e26968ca3cc0a1f06a29cd54913beb662ff
[2]: PKGBUILD.diff
Comment by loqs (loqs) - Saturday, 04 November 2023, 15:06 GMT
Ah I think I see what is wrong. When skipping the venv /usr/share/pwndbg/pwndbg/ needs to be added to the sys.path.
Comment by loqs (loqs) - Saturday, 04 November 2023, 15:16 GMT
Attached updated patch. The pwndbg-2023.07.17-1.3-any.pkg.tar.zst built from applying the attached patch plus the capstone 5 packages works for me at least to the point of gdb loading pwndbg without a venv.
Comment by berilac (berilac) - Saturday, 04 November 2023, 15:34 GMT
@loqs Beautiful. Can confirm this latest patch means I can now run gdb and import successfully, without a `venv` and without setting `PWNDBG_PLEASE_SKIP_VENV`.
But of course only with the `v5` binaries for `capstone` and `python-capstone`.
So presumably we now only need wait for those packages to be updated in the official repo?