Skip to content

Commit

Permalink
Deprecate to_requested_schema (apache#918)
Browse files Browse the repository at this point in the history
* deprecate to_requested_schema

* prep for release
  • Loading branch information
sungwy authored Jul 12, 2024
1 parent a3dd531 commit b11cdb5
Show file tree
Hide file tree
Showing 2 changed files with 35 additions and 0 deletions.
15 changes: 15 additions & 0 deletions mkdocs/docs/how-to-release.md
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,21 @@ The guide to release PyIceberg.

The first step is to publish a release candidate (RC) and publish it to the public for testing and validation. Once the vote has passed on the RC, the RC turns into the new release.

## Preparing for a release

Before running the release candidate, we want to remove any APIs that were marked for removal under the @deprecated tag for this release.

For example, the API with the following deprecation tag should be removed when preparing for the 0.2.0 release.

```python

@deprecated(
deprecated_in="0.1.0",
removed_in="0.2.0",
help_message="Please use load_something_else() instead",
)
```

## Running a release candidate

Make sure that the version is correct in `pyproject.toml` and `pyiceberg/__init__.py`. Correct means that it reflects the version that you want to release.
Expand Down
20 changes: 20 additions & 0 deletions pyiceberg/io/pyarrow.py
Original file line number Diff line number Diff line change
Expand Up @@ -156,6 +156,7 @@
from pyiceberg.utils.concurrent import ExecutorFactory
from pyiceberg.utils.config import Config
from pyiceberg.utils.datetime import millis_to_datetime
from pyiceberg.utils.deprecated import deprecated
from pyiceberg.utils.singleton import Singleton
from pyiceberg.utils.truncate import truncate_upper_bound_binary_string, truncate_upper_bound_text_string

Expand Down Expand Up @@ -1279,6 +1280,23 @@ def project_batches(
total_row_count += len(batch)


@deprecated(
deprecated_in="0.7.0",
removed_in="0.8.0",
help_message="The public API for 'to_requested_schema' is deprecated and is replaced by '_to_requested_schema'",
)
def to_requested_schema(requested_schema: Schema, file_schema: Schema, table: pa.Table) -> pa.Table:
struct_array = visit_with_partner(requested_schema, table, ArrowProjectionVisitor(file_schema), ArrowAccessor(file_schema))

arrays = []
fields = []
for pos, field in enumerate(requested_schema.fields):
array = struct_array.field(pos)
arrays.append(array)
fields.append(pa.field(field.name, array.type, field.optional))
return pa.Table.from_arrays(arrays, schema=pa.schema(fields))


def _to_requested_schema(
requested_schema: Schema,
file_schema: Schema,
Expand Down Expand Up @@ -1434,6 +1452,8 @@ def field_partner(self, partner_struct: Optional[pa.Array], field_id: int, _: st

if isinstance(partner_struct, pa.StructArray):
return partner_struct.field(name)
elif isinstance(partner_struct, pa.Table):
return partner_struct.column(name).combine_chunks()
elif isinstance(partner_struct, pa.RecordBatch):
return partner_struct.column(name)
else:
Expand Down

0 comments on commit b11cdb5

Please sign in to comment.