Changelog
All notable changes to this project will be documented in this file.
v8.0.2 (2024-08-01)
Changes for this release are only about our CI pipeline.
We fixed the bug with the tag name for the docker image: it was v8-0-1 in the previous release instead of v8.0.1.
v8.0.1 (2024-07-24)
fix
revert commit 395f294ed2cd38125ed132fdc13289e2870aa6d0 that breaks the process launching on windows
v8.0.0 (2024-07-16)
BREAKING CHANGE
The
fraction
parameter ofconvert
(and the--fraction
cli flag) has been removed, because we didn’t actually use it in the codeThe . folder by default is now /data on docker image, please update your mounts accordingly.
We replaced the home-made gltf support by pygltflib. It should be compatible with the previous usage for the most part, except for direct usage of
gltf.py
(replace it with direct usage of pygltflib) andB3dm.from_numpy_arrays
, which has been replaced byB3dm.from_primitives
. We believe this will be vastly easier to use. Under the hood, it generates gltf primitives inside the tile.the
all
section in setup.py has been removed becausepyproject.toml
doesn’t support creating them from all the other sections, please usepip install -e .[postgres,las,ply]
instead.TileContent.print_info(), B3dm.print_info() and Pnts.print_info() have been removed and replaced by str methods. By assuming
b3dm_tile
is a B3dm tile, one may access to string representations with print(b3dm_tile) or str(b3dm_tile).Tileset.root_uri
must now be set after the constructionBoundingVolume
: the functionsis_box
,is_region
andis_sphere
from this class are removed, useisinstance(BoundingVolumeBox)
(or the relevant subclass) instead.
Feat
the docker image is now also pushed on dockerhub
the home-made gltf support is replaced by pygltflib. a
gltf_utils
module has been introduced to provide a higher-level API to pygltflibb3dm: support batchids with pygltflib
b3dm: min/max attributes in gltf accessors
b3dm: support materials & uvs with pygltflib
b3dm: from_gltf also takes a feature table
pnts: display info about batchtable when printing
support intensity in pointcloud conversion, only in uint8 at the moment. Intensity of las files are converted from uint16 to uint8 at the moment.
tile.py: add method to transform coords according to this tile transformation property
convert.py: accept an existing out folders if empty
bounding_volume_box: add classmethod .from_points and .from_list and document a bit
We now have a logo!
tileset: add mechanism to support extensions
base_tiler: create an abstract tiler. This abstract class aims at making the creation of a custom tiler easier.
Fix
tile.py: write content URI as posix
typing: quantized positions in pnts are uint16, not uint8
export: fix optional import of psycopg2
tile.py: make sure we reshape and flatten in column order
py3dtiles/tileset/content/b3dm.py: sync the tile info after array initialization
buildings.b3dm fixture is outdated
BatchTable: ignore extensions in from_array
point_tiler: set refine mode in the correct place
Tile: don’t raise error if file missing
docker: move the workdir to /data in images
convert.py:
don’t crash on trying to remove non-existent path
fix transmission of worker error messages to main thread
fix(b3dm.py): make node_matrix serializable
Refactor
convert.py: use our Tile and Tileset classes instead of dicts
tileset.py: remove root_uri from constructor parameters
bounding_volume_box: make sure transform accept only 4x4 matrices
make export.py independant from home-made gltf
print_info is replaced by str properties for each tile_content
convert.py:
remove unused parameter
simplify Process declaration
move tiler point specific code outside of convert.py by using the abstract tiler
move pytest fixtures into tests/conftest.py
Miscellaneous
for nix users, a shell.nix has been added to the project.
v7.0.0 (2023-12-12)
Community change: Big news!
We welcome Lorenzo Marnat (Liris) as a new maintainer! The Liris has contributed a lot to py3dtiles through their fork and the client application py3dtilers
and Lorenzo is currently helping getting these improvements merged. This is a very important step forward developing our community and we’re very glad to have him in the team.
As a result, we have made it so that py3dtiles is less tied to Oslandia and more to its own community. We hope to make contributions and implications even easier and open to other entities:
the repository has moved to its own organization: https://gitlab.com/py3dtiles/py3dtiles and the main branch is now
main
instead ofmaster
. The existing issues and merge requests has been kept and the branch they point to has been updated. Please update your git remotes, especially because there is no redirection at the git level: we needed to recreate the py3dtiles on oslandia namespace to setup a redirection from the old pages.The main domain is now
https://py3dtiles.org
. A redirection has been made, but we still advise everybody to update their bookmarksWe have a
GOVERNANCE.md
documentation that describes how the community will operate from now on.and we now have a chat room on matrix.org!
BREAKING CHANGE
FeatureTable*
has been renamed toPntsFeatureTable*
py3dtiles now requires numpy >= 1.24 to allow python 3.11 support (note: 3.12 support must wait for the next numba release)
Big change in the way we deal with dependencies. With more and more file format supports added, the dependency tree is becoming bigger and bigger. To avoid cluttering client environments with dependencies for formats they don’t need, we are now using the extras_require
section for some formats. Here is the list of file formats and the command to use to get the corresponding dependencies:
postgresql support:
pip install py3dtiles[postgres]
las/laz support:
pip install py3dtiles[las]
(note: laz support is still conditioned by the presence of laszip or lasrs, see installation documentation)ply support:
pip install py3dtiles[ply]
The .[all]
sections allows to get all these dependencies. The docker image contains everything.
Csv and xyz support depends on the standard library, therefore they are always included.
Feat
add support for python 3.11
add featureTable and B3dmFeatureTable classes
upgrade numpy
support for csv files and classification data in the xyz_reader
Fix
mypy: make type declarations compatible with python3.8
convert.py: ensure the main process waits for all child to finish
align b3dm_feature_table to 8 bytes
Pnts from_file method to ensure TileContent is PNTS
PNTS ft body to_array returns one array
distance.py: remove a numba warning
docker: create it on bullseye to have liblaszip8
Refactor
convert.py: change self.idle_clients type to set
reorder imports according to our isort config
rename PNTS FeatureTable classes
v6.0.0 (2023-04-04)
BREAKING CHANGE
py3dtiles merge
now takes tileset path instead of a folder path.The class
BaseExtension
becomes an abstract class. Please use one of the subclassesTileContentReader
class has been removed and its static methods are now plain functions intile_content_reader.py
.The
read_file
function has been renamed toread_binary_tile_content
instance created with FeatureTableHeader.from_semantic method.the
queue
argument is removed from readerrun
method signatures (these functions do not send messages anymore)The method
sync
of thePntsBody
andB3dmBody
has been moved toTileContent
class. The parameterbody
has been removed.The
Pnts.from_feature
method signature has completely changed. Numpy data type isn’t used anymore but aFeatureTableHeader
The attributes:
B3dm.from_glTF
has been renamed toB3dm.from_gltf
B3dmBody.from_glTF
has been renamed toB3dmBody.from_gltf
B3dmBody.glTF
has been renamed toB3dmBody.gltf
Few import changes:
Change
from py3dtiles.tileset.tile_content_reader import read_file
tofrom py3dtiles.tileset.content import read_file
Change
import py3dtiles.tileset.batch_table
toimport py3dtiles.tileset.content.batch_table
Change
import py3dtiles.tileset.content.feature_table
inimport py3dtiles.tileset.content.feature_table
The import of
TileContent
has been changed, now it is :from py3dtiles.tileset.content import TileContent, TileContentBody, TileContentHeader
The type
ThreeDDictBase
has been renamed toRootPropertyDictType
The class
Extendable
has been renamed toRootProperty
and :its method
add_extension
has been deleted, add directly to the attributeextensions
its method
has_extensions
has been deleted, check directly the content of the attributeextensions
its method
get_extension
has been deleted, retrieve directly the extension with its name in the dict attributeextensions
its method
get_extensions
has been deleted, get directly the attributeextensions
The class
Tile
has been modified:The
get_content
method has been renamed toget_or_fetch_content
. If thecontent
is already loaded or thecontent_uri
is absolute,root_uri
must be None.The
set_content
method has been removed, set directly the attributetile_content
The
set_content_uri
and Tileget_content_uri
methods have been removed, set and get directly thecontent_uri
attribute.The method
get_children
has been renamed toget_all_children
The method
get_direct_children
has been deleted, use directly the attributechildren
to get them. Note: It is highly recommended to still use the method Tile.add_children (and not children.append)The method
has_children
has been removed, check directly if the attributechildren
is empty.
The class
TileSet
has been modified:The
from_dict
requires a new parameter:root_uri
, that is the folder where the tileset is.The method
add_asset_extras()
has been deleted, use directly the attributeextra
(dict)The classes for tile content have been modified:
The class
TileContent
is now an abstract class, useB3dm
orPnts
class insteadTo create a
Pnts
orB3dm
instance, you must indicate the header and the body instance in the constructorThe
TileContentType
class has been removedThe type attribute of the class
TileContentBody
has been removedThe class
Feature
has been removedThe
FeatureTableHeader.from_dtype
method has been replaced byfrom_semantic
The
positions_dtype
,colors_dtype
andnormal_dtype
attributes of the classFeatureTableHeader
has been removed. To get the data type, useSEMANTIC_TYPE_MAP[semantic]
with semantic eitherfeature_table_header.positions
,feature_table_header.colors
orfeature_table_header.normal
The
FeatureTableBody.positions_arr
attribute has been renamed toposition
The
FeatureTableBody.colors_arr
attribute has been renamed tocolors
The
FeatureTableBody.positions_itemsize
has been removedThe
FeatureTableBody.colors_itemsize
has been removedThe
FeatureTableBody.from_features
method has been removed, useFeatureTable.from_features
insteadThe
FeatureTableBody.positions
andFeatureTableBody.colors
methods have been removed, useFeatureTable.get_feature_color_at
andFeatureTable.get_feature_position_at
insteadThe
FeatureTable.feature
method has been renamed toget_feature_at
and return tuple instead of Feature instanceThe signature of
FeatureTable.from_features
has completely changed. Numpy data type isn’t anymore use, but aFeatureTableHeader
instance created withFeatureTableHeader.from_semantic()
method
Feat
Windows is fully supported
a docker image is now built on each release
Classification data can now be exported in the tileset
Add an universal merger
Load tileset from dictionary with lazy tile content loading
Add methods to remove tilesets and tiles on disk
Add the support of extras and extensions properties for Asset, BoundingVolume, Tile and TileSet
Import and export extensionsUsed and extensionsRequired (TileSet)
tileset.py: Sync root uri of tiles when the tileset is written as json
tileset.py: Add methods
from_file
andget_tile_contents
tile.py: The transformation attribute can be set in
__init__()
bounding_volume.py: Add new methods as abstractmethod
new Py3dtiles exceptions have been created:
TilerException
ThreeDTileSpecError
PntsSpecError
B3dmSpecError
TilesetSpecError
BoundingVolumeMissingException
Use transformation when sync bounding volume
feature_table.py: Improve the support of pnts feature table
batch_table.py: Allow mix of json and binary data
Create a dedicated type for the export of the feature table header
Add the Batch Table Hierarchy extension
Add get_points method to PntsBody
Add the from_points method in Pnts class
Publish number_of_points_in_tileset on the API
Typing work largely completed
Fix
api.rst: Correct the examples in the api doc
Change gltf padding to fix b3dm body alignment
las_reader: Correct color_scale calculation
convert.py: Allow
convert
to work with thespawn
multiprocess methodReverse the recursivity to prune from leafs to root instead of root to leafs
allow multiple user to convert simultaneously on the same machine
fix a warning about a 0 division that can occur when aabb have a 0-size dimension.
Refactor
Remove useless llvm dependency
Retrieve crs with laspy instead of pdal and remove the pdal dependency
Rename
read_file
function toread_binary_tile_content
Move
print_b3dm_info
andprint_pnts_info
in thetile_content
classesuse tile.content_uri in write_to_directory method
merger.py: use tileset module instead of using custom tools
bounding_volume_box.py: improve the readability of some methods
use fix size numpy data type (
np.uint8
instead ofnp.ubytes
)explicit re-export module with a better API import style
move all node_process functions in a class to clean/simplify code
move the main node processing loop to
convert.py
for centralizing message sendingyield node process and send message in the main node processing function
Remove useless try..except blocks
halt_at_depth
defined starting from node name lengthRemove
node_catalog
parameter ofinsert
and _split
methods since it is useless.yield node process and send message with yielded values [WIP]
Reader
run
methods now yields tuples of coordinate, color and classification arrays and the message is sent by the Worker objectpnts_writer.py:
run
method now yields the total amount of nodes and the message is sent by the Worker objecttransform
TileContent
class to abstract class and add typing infotileset/utils.py: The static methods in
TileContentReader
become functions in a dedicatedtile_content_reader.py
moduleLowercase all
glTF
occurrencesnode.py: Rename
reminder
occurrences toremainder
v5.0.0 (2023-02-02)
BREAKING CHANGE
py3dtiles no longer support python 3.7 (it reaches EOL in june)
The function convert_to_ecef has been removed
Many imports have been changed, please update them accordingly:
from py3dtiles import B3dm
becomesfrom py3dtiles.tileset.content import B3dm
from py3dtiles import GlTF
becomesfrom py3dtiles.tileset.content import GlTF
from py3dtiles import Pnts
becomesfrom py3dtiles.tileset.content import Pnts
from py3dtiles import BatchTable
becomesfrom py3dtiles.tileset.batch_table import BatchTable
from py3dtiles import BoundingVolumeBox
becomesfrom py3dtiles.tileset.bounding_volume_box import BoundingVolumeBox
from py3dtiles import Extendable
becomesfrom py3dtiles.tileset.extendable import Extendable
from py3dtiles import Extension
becomesfrom py3dtiles.tileset.extension import BaseExtension
from py3dtiles import Tile
becomesfrom py3dtiles.tileset.tile import Tile
from py3dtiles import TileContent
becomesfrom py3dtiles.tileset.tile_content import TileContent
from py3dtiles import TileSet
becomesfrom py3dtiles.tileset.tileset import TileSet
from py3dtiles import TileContentReader
becomesfrom py3dtiles.tileset.utils import TileContentReader
from py3dtiles import TriangleSoup
becomesfrom py3dtiles.tilers.b3dm.wkb_utils import TriangleSoup
from py3dtiles import Feature
becomesfrom py3dtiles.tileset.feature_table import Feature
The BoundingVolume class become an abstract class
The way of getting / setting
transform
has changed:The methods
set_transform
andget_transform
of the classTile
has been removed, and the attributetransform
of the classTile
has been renamed from_transform
totransform.
Please use the attribute directly.in the same vein, the method
set_transform
of the classTileset
has been removedThe method set_from_array from the class BoundingVolumeBox has been removed, use set_from_list instead (with an ArrayLike)
some parameters types have been set to always be a flat numpy array of
np.float64
:Tile.transform
The parameter
offset
of the methodBoundingVolumeBox.translate
The parameter
transform
of the methodBoundingVolumeBox.transform
The parameter
box_list
of the methodBoundingVolumeBox.set_from_list
The parameter
mins_maxs
of the methodBoundingVolumeBox.get_box_array_from_mins_maxs
The parameter
points
of the methodBoundingVolumeBox.get_box_array_from_point
Features
convert.py: add parameter to ignore CRS mixin in input files
add the import and export of batch table from pnts (only json)
reader/ply_reader.py: support .ply files
typing.py: create the tileset json structure with typing annotations
Fixes
add and update typing annotations to fix all mypy issues
check magic value as bytes not str
exclude B028 flake8 rule
node.py: delete tile children when the tile root is moved in a sub tileset
Refactor
remove all asserts in source code
select only used fonctions from transformation file and update the code
LICENSE: remove Mapbox third party license
remove copied earcut code and use mapbox library
change file hierarchy
v4.0.0 (2023-01-09)
BREAKING CHANGE
The parameter
srs_in
andsrs_out
ofpy3dtiles.convert.convert
have been renamed tocrs_in
andcrs_out
. Furthermore, their type is no longer int or str butpyproj.CRS
. No change has been made to the command-linepy3dtiles convert
, but you can use proj4 string in addition to epsg code. To migrate old code, instead of:
from py3dtiles.convert import convert
# ...
convert('without_srs.las', outfolder=tmp_dir, crs_out='4978')
you can do:
from pyproj import CRS
from py3dtiles.convert import convert
# ...
convert('without_srs.las', outfolder=tmp_dir, crs_out=CRS.from_epsg(4978))
SrsInMissingException has been moved from
py3dtiles/utils.py
topy3dtiles/exceptions.py
.
Feat
The main feature of this release is that you can now mix las/laz/xyz files in one invocation of the convert function.
Fix
shared_node_store.py: fix node removing in cache if already deleted
change srs_in to srs_out to fix refactoring error
avoid duplicate points when the mode is replace
Refactor
convert: rename the ‘infos’ variable and function to ‘file_info’
convert: use CRS almost everywhere instead of string representing epsg code
exceptions: move all custom exceptions at the same place
convert: use a dictionary to find the correct reader
fix issues find by pre-commit
Chores
add pre-commit hooks
v3.0.0
BREAKING
Some renaming has been done to better follow the 3dtiles specification:
TileHeader
->TileContentHeader
TileBody
->TileContentBody
The API of merger.py::merge wasn’t really convenient to use and has now changed. Now the signature is:
def merge(folder: Union[str, Path], overwrite: bool = False, verbose: int = 0) -> None
the argument verbose of the cli interface has changed. To increase the verbosity, the number of -v is counted (-vv will be a verbose of 2).
Boolean options has been changed from
--foo=1
to simple flags:--foo
. Affected options are--overwrite
and--graph
.--rgb=no
has been replaced by a--no-rgb
option to deactivate it. The default is still to keep color information
Features
support laz if laszip is installed
windows support (NOTE: testers needed)
Some classes to represent 3Dtiles concepts have been added:
BoundingVolumeBox
TileSet
Tile
uExtension
Fix
The geometric error of two merged tilesets is now the biggest of the two tileset geometric error divided by the ratio of kept points. We believe this use of GeometricError fits more the spirit of the specification.
node: avoid empty children array in tileset.json
setup.py: fix missing dependency pytest
node: avoid to add empty children array
disable padding if already 8-byte aligned instead of adding 8 new empty bytes
featureTable: add a 8-byte boundary for FeatureTableBody
featureTable: change the boundary from 4 to 8
replace sys.exit(1) in convert by raising an exception
v2.0.0
This releases completely reworks py3dtiles command line and add new features.
The command line now uses subcommands syntax, in order to expose a single entry point and support multiple commands. The existing commands ‘export_tileset’ and ‘py3dtiles_info’ became ‘py3dtiles export’ and ‘py3dtiles info’.
Changes
relicensed as Apache 2.0.
minimal python supported is now 3.8
dependencies versions has been updated:
laspy should be at least 2.0
numpy at least 1.20.0
Tile has been renamed to TileContent
Features
New features were added, the main one being: py3dtiles can be used to convert pointcloud las files to a 3dtiles tileset.
This is the purpose of the ‘py3dtiles convert’ command. It supports multicore processor for faster processing, leveraging pyzmq, and the memory management has been carefully implemented to support virtually unlimited points count.
Other features:
read points from xyz files
Documentation are now published at https://oslandia.gitlab.io/py3dtiles
Fixes
53580ba Jeremy Gaillard fix: use y-up orientation for glTF objects in export script
65d6f67 Jeremy Gaillard fix: proper bounding box size in export script
3603b00 Augustin Trancart fix: reliably select triangulation projection plane and orientation
fd2105a jailln Fix gltf min and max value