Other Technology

Esri Announces LAS Compression

I was told that Esri was working on their own version of compression for LAS at last summer’s Survey Summit. It now appears to be documented based on this update sent out in Decmber. The compressed file has an extension of .zlas. If you have any experience in working with it I am sure some folks will be interested in hearing about it.

27 Comments

  • Well, there is a Help file reference to a new scheme but they have not, to my knowledge, made any formal announcements.

  • A little research on the topic yielded the following;
    “Esri has developed compression technology for LAS format LiDAR that reduces file size significantly more than generic compressors can (e.g., gzip).” No mention of .laz comparison though…

    A little more digging turned up;
    Lossless compression. The lidar point record data is preserved exactly. There is no loss of information, which is critical if you want to retain integrity of the source data. Compressed data does not need to be decompressed prior to use with ArcGIS 10.2.1. Statistics and spatial indexing are also added during the compression process. Lastly, Esri plans to make this compression/decompression utility available as a free executable via download from esri.com

  • If the format is not fully documented, it’s not an open format. Dealing with closed formats is a pain point for me that api-only access does not resolve.

  • This is really disappointing and reflects poorly on ESRI.

    (1) The days of proprietary file formats are long gone. In the geospatial market (and others), locked formats provide the vendor with short and possibly medium term wins, but cause long term incompatibilities and fragmentation.

    (2) The world already has LAZ, and it’s not clear to me yet that ESRI’s format provides any significant advantages over LAZ.

    (3) Despite ESRI’s continual posturing about openness and interoperability and community, I’ve not heard any word that ESRI attempted to engage the community before attempting fragmenting the market like this.

    -mpg

      • Wrong. Please read the license file more carefully. The restriction you mention has absolutely nothing to do with the LASzip compressor or the LAZ format. The LASzip LiDAR compressor has a standard LGPL license. After all, if was funded in parts by the United States Army Corps of Engineers (USACE) because the federal government wanted to avoid at the time that a proprietary format (namely LizardTech’s LiDAR Compressor) would become a de-facto standard …

    • So what exactly is the LAZ format? Here are a few questions for the experts:

      1. Is the format well documented, like the LAS specification?

      2. Can any programmer implement a compressor/de-compressor from scratch by following the specification, such that the compressed result can be correctly decoded by another independent implementation (like what anyone can do with LAS)?

      3. Can he do so without any constraint or legal implication?

      4. If there is a need for enhancement/extension, is there a committee that someone can go to? What are his options if he needs LAS 1.4 compression support?

      • Question 1: Yes. There is an ASPRS journal publication and – unlike for the LAS format – a reference implementation: http://laszip.org.

        Question 2: When was the last time you implemented a JPEG compressor? There is aforementioned paper and open source code, specification . The key point is that it is open and be verified, improved upon, or adapated by anyone.

        Question 3: Yes. The license is standard LGPL.

        Question 4: The committee is us and we are functioning quite well in contrast to the official LAS committee. For example, want multi-threaded compression or decompression – be my guest, want spatial indexing included in the file – certainly an easy add on, want LASzip extended to LAS 1.4 … well … USACE helped to pay last time (it was cheap). Maybe NOAA, USGS, or another countries National Land Survey could step up this time …

        PS: Who are you? Would be nice to have an open discussion.

        • So there is no such thing as a LAZ open format specification that everyone can implement. Instead, there is an implementation/library that you maintain and control. Here are some more questions:

          1. Is it correct that everyone must ask/hire you if they need to do anything with LAS compression and/or need to make any changes?

          2. If there are issues with your implementation/library, who will investigate and fix them?

          3. You don’t allow any competition because you already have one?

          • You need to do some reading on what LGPL means if you suggest I control anything. The reason we chose LGPL in the first place was to prevent a big company to release a proprietary “LAZ clone” and squish our open efforts with their market might.

            Are you part of the same PR team that tried to discredit ERDAS when they had a disagreement with ESRI way back then?

          • Your first point is a serious misunderstanding of what open source software is. No one *controls* the library, and any and all are free to do whatever they want with it subject to the terms of the LGPL license. Those terms require that you *also release* the source to those changes as part of any distribution effort of the software.

            Anyone is free to fix and investigate issues. The source code is there for anyone to dive in and figure out any edge case or reproduceable bug. A proprietary SDK user does not have that luxury. Most users of open source software never need that feature either — but it sure is nice when it is available.

            Finally, competition is great when it’s fair.

        • To suggest that “the committee is us” and link to http://groups.google.com/group/lasroom is disingenuous Martin.

          Yes, there is indeed an ongoing discussion in that group. But from what I understand, messages sent to that group need to be approved by a moderator before they appear. Who are the moderators? How can we find out who the moderators are? What are their commercial affiliations? How are the moderators selected?

          I’d suggest that the only moderator is Martin Isenburg himself.

          Personally I’d like to see a more open “committee” where posts can be seen and answered by all without the need for approval by a middle-man with commercial interests.

          • You have a point. I originally created this group as a public discussion to go on record with suggestions or concerns about the LAS specification – as I felt that the LAS committee was missing a way for the community to have input in the ratification process of LAS 1.4. I added moderation when debates got a little too heated. I did not let through the occasional one-liner that was only anger but no content and the odd spam message.

            It seemed that the community was quite content with the way I was operating it. Really … had I been “silencing” some actual discussion content folks would surely have protested loudly via other venues.

            That said with the forum having evolved into a permanent LAS discussion forum it may a good idea to have a team of moderators. You seem to feel strongly about it. Care to join?

          • W2 made a valid point about “The LAS room” being moderated only by one person. We now have three moderators, Gottfried (academia, Austria), Al (government, USA), and me (open source/industry, Everywhere). See “The LAS room” for more details …

  • It may require a lawyer to understand the language. Why the restrictions at all (especially if the project was funded by the US government)?

  • I’d like to thank W2 for an interesting discussion – By what name should we address you?

    Seconding Martin’s suggestion for a good reading of the LGPL which would answer a number of your questions. Here’s good start – bit.ly/1hoGuW5

  • I think a good idea would be to ask the commercial community that is already using LAZ within their software. They clearly have an understanding of the LGPL licensing requirements.

    A couple points:

    1). LAZ is based solely on the LAS specification.
    2). There is a peer reviewed publication outlining how the compression algorithms work. Clearly ESRI has taken note of this.
    3). There is an open-source software repository available to all for implementation or additional functionality.
    4). LAZ was funded by the USACE Cold Regions Research & Eng. Lab as part of a larger effort to bring “important” tools to the geospatial community that are seen as essential. The same as it is doing for PDAL and many other libraries. Having the code as LGPL helps preserve heritage of where code comes from. Basically, the government is tired of paying twice for something it already paid for.

    Want to use LAZ, please do, it not, feel free to use something else that works for your purpose. We do feel the community agrees LAZ is a step in the right direction and a number of government agencies have adopted it as its primary format, mainly because it adheres to the LAS specification.

  • Thank you for those who provide answers and links. They for sure help, though some of my specific questions remain unanswered.

    W2 was used as a screen name in response to MPG which I believed is also a screen name.

    • In the interests of full disclosure: I am Michael P. Gerlek, mpg@flaxen.com. A number of the people on this thread know me and my past involvements with both the proprietary and open file format worlds.

      -mpg

  • To Martin: So far I believe that my questions are valid and relevant. To suggest that I am part of a PR team is unacceptable.

    In fact, you are the one who has been unfairly discrediting your competitor without any sound evidence. Just because zLAS has a similar compression performance, must it be a “clone” of LAZ? Is this a fair and valid conclusion?

    Obviously, zLAS is much larger in scope than LAZ according to https://blog.lidarnews.com/content/view/10214/, and compression is just a part of it. To me, zLAS looks more like a spatial database solution where metadata, indexing, as well as data compression are the three essential components.

    • I wasn’t saying code was copied. The performance of LAZ was cloned, spawned on multiple threads, enhanced with the functionality of LAX, and combined with a standard spatial sort (available, for example, in lassort.exe) to improve locality. This can be implemented within a few hours to days based on LASzip or LASlib while also remaining fully compatible to the existing already widely-adopted open standard. We would have gladly offered our services to create something better that works for both ESRI and the LiDAR community … at a fraction of the cost.

      Instead a costly development effort was carried out – without involving any of the stakeholders in the LiDAR community – to create something that brings this very community only three new features: being differemt, being closed, and being ESRI. Why fragment the market? Why not involve us in this process? Why not embrace LAZ for the benefit of all? Why not follow through on – as ‘mpg’ (a well-known developer’s nickname) put it – the “continual posturing about openness, interoperability, and community”?

      Here a good read on the topic by Paul Ramsey …

      PS: Sorry about the PR team thing. You are right, that was uncalled for.

      • I’m not going to weigh in on the technical stuff because I barely use LiDAR but appreciate open development, standards, formats. My thoughts have nothing to do with ESRI or LAStools because I don’t use either, just feelings about the paradigm of openness.
        Let me also appreciate someone who seems to know how community and “open” works. Specifically thank you that when…

        Someone doesn’t like the way the forum is set up, you explain the history, agree, and offer a “seat at the table”.

        You made a comment that was perceived as a little touchy, were called on it, and apologized without caveats.

        Your arguments primarily are based in the desire for open standards and the proof is in the pudding as they say in the licensing and availability of code.

        Just a personal wish but I wish vendors, operating systems, and programs would compete based on service, usability, and features, not by how prevalent our format is and how many people are locked into a product because they are locked into a format (see office suites for an example).

      • Thank you for clarifying what you meant by “clone.” It did appear to me that you were accusing them copying code, particularly with your comments of wanting to have a peek on their code (along with a fake leak-site).

        I recall that a while ago you were speculating about the possible technical reasons for which they don’t (or cannot) use LAZ (see below). Would this still be a possibility?

        “Anyone want to speculate what other technical reasons there might be to not use (an adapted version) of LAZ …? Maybe they seperate out the compression of those parts of the per-point data that does not usually get visualized (e.g. GPS time) into a different bands so it does not have to get decompressed unless needed? That would seem reasonable. That is – in fact – an idea I have already been playing with for when LASzip gets extended to the new 1.4 point types.”

Leave a Comment

This site uses Akismet to reduce spam. Learn how your comment data is processed.