FS#70162 - [python-pyarrow] Unable to load feather files from Pandas

Attached to Project: Community Packages
Opened by Keith Hughitt (khughitt) - Thursday, 25 March 2021, 19:20 GMT
Last edited by Bruno Pagani (ArchangeGabriel) - Sunday, 17 October 2021, 01:48 GMT
Task Type Bug Report
Category Packages
Status Closed
Assigned To Bruno Pagani (ArchangeGabriel)
Architecture All
Severity Low
Priority Normal
Reported Version
Due in Version Undecided
Due Date Undecided
Percent Complete 100%
Votes 0
Private No

Details

Description:

When PyArrow is installed, pandas usually has support for reading feather, parquet, etc. files.

With the current versions of pyarrow/feather installed, however, I get an error when attempting to load such a file, e.g.:

```
import pandas as pd
x = pd.read_feather("test.feather")
```

**Error**:

```
ImportError Traceback (most recent call last)
<ipython-input-2-614a727ea0ba> in <module>
----> 1 x = pd.read_feather("/data/clean/geo/1.1/GSE2912/GSE2912_gene_expr.feather")

/usr/lib/python3.9/site-packages/pandas/io/feather_format.py in read_feather(path, columns, use_threads, storage_options)
118 type of object stored in file
119 """
--> 120 import_optional_dependency("pyarrow")
121 from pyarrow import feather
122

/usr/lib/python3.9/site-packages/pandas/compat/_optional.py in import_optional_dependency(name, extra, raise_on_missing, on_version)
113 minimum_version = VERSIONS.get(name)
114 if minimum_version:
--> 115 version = get_version(module)
116 if distutils.version.LooseVersion(version) < minimum_version:
117 assert on_version in {"warn", "raise", "ignore"}

/usr/lib/python3.9/site-packages/pandas/compat/_optional.py in get_version(module)
54
55 if version is None:
---> 56 raise ImportError(f"Can't determine version for {module.__name__}")
57 return version
58

ImportError: Can't determine version for pyarrow
```


Packages used:

- python 3.9.2-1
- python-pandas 1.2.3-1
- python-pyarrow 3.0.0-1
- arrow 3.0.0-1


Steps to reproduce:

(see above)
This task depends upon

This task blocks these from closing
 FS#70611 - Missing submodules in python-pyarrow package 
Closed by  Bruno Pagani (ArchangeGabriel)
Sunday, 17 October 2021, 01:48 GMT
Reason for closing:  Fixed
Additional comments about closing:  Done in 5.0.0.
Comment by Bruno Pagani (ArchangeGabriel) - Friday, 26 March 2021, 08:29 GMT
Actually it seems the pyarrow package is empty from a python pov: there is nothing to import at all from pyarrow. I’m not sure why though, and can’t investigate currently.

Loading...