tag:blogger.com,1999:blog-26400036668917287242024-02-14T10:36:03.066+01:00Tomaz's dev blogTomaž Vajngerlhttp://www.blogger.com/profile/17183461757847157603noreply@blogger.comBlogger20125tag:blogger.com,1999:blog-2640003666891728724.post-17120058072576847842022-09-15T15:18:00.005+02:002022-09-15T15:19:40.371+02:00Chart Data Tables<p>Chart data table is a feature of charts, that presents in the chart area a data table with the values that are visualised by the chart. The data table is positioned automatically at the bottom of the chart, and can for certain chart types replace the X-axis labels. Until now this feature has been missing in LibreOffice, but thanks to the funding of NGI, it is now implemented.</p><table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="float: left; margin-right: 1em;"><tbody><tr><td style="text-align: center;"><img border="0" data-original-height="541" data-original-width="811" height="48" src="https://blogger.googleusercontent.com/img/a/AVvXsEgx-whNTCTOQ9YbFy1qJ5Y275pTjYvdSFQ1sAz6lFrbthw5K3jrMoYExvpTAdsfx2Dl8SMX94LaHwBt0bUL0JNKBJY9NnNhAJX6XeOAp2zm_FhbRajJBD8sl6BvL2Y66ysjwozdcocWCyOvcpMiHmitJvz8f5hO5eRxhSbzzD4xloNfeYdYLkbRa7qn4Q=w72-h48" style="margin-left: auto; margin-right: auto;" width="72" /></td><td class="tr-caption" style="text-align: center;"><div style="text-align: left;"><br /></div><div style="text-align: left;">This project has received funding from the European Union’s Horizon 2020 research and innovation programme under grant agreement No 871498</div><div><div style="text-align: left;"><br /></div></div></td></tr></tbody></table><div><br /></div><p><br /></p><p><br /></p><table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto;"><tbody><tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhTDTul1oyTwfQ7X08CCBHcvHi-X_bK0zdc4xHVAzlx0vNIIb0AOgBmmqaQaRhCB7fSZ3mFNCLiq6hK9TIOPvUlyPiGlLh5x5mXmKkGKGSRllHhOwK2nsS6qoGkbmn_-yumkqqE8F9xcQSwF-8bFesjI10vG5aPcwrmPE_nYYwskevLSWP92E0n5Lk5aw/s1428/DataTablesExample.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="887" data-original-width="1428" height="398" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhTDTul1oyTwfQ7X08CCBHcvHi-X_bK0zdc4xHVAzlx0vNIIb0AOgBmmqaQaRhCB7fSZ3mFNCLiq6hK9TIOPvUlyPiGlLh5x5mXmKkGKGSRllHhOwK2nsS6qoGkbmn_-yumkqqE8F9xcQSwF-8bFesjI10vG5aPcwrmPE_nYYwskevLSWP92E0n5Lk5aw/w640-h398/DataTablesExample.png" width="640" /></a></td></tr><tr><td class="tr-caption" style="text-align: center;">Figure 1: Charts with a data table</td></tr></tbody></table><br /><h2 style="text-align: left;">Chart data table usage and description</h2><table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto;"><tbody><tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh4ITFw_3UzBlgSBFxYC0_OsKVWEntVsDZK9wi0zrzabNeJ39GgZ8LrL-z9LN7PgKw8b5CehDgkf5dCdTygr16VUddj2M2pvJKUhNmKOz_fyBhDIBPmSOpQHTgOcjOvBoH6NkieE0Q7qdYqW22T8a7Wt2Vhvjuy436KEI18alDlnETee0azfreEET4_cA/s340/InsertDataTableDialog.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="339" data-original-width="340" height="319" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh4ITFw_3UzBlgSBFxYC0_OsKVWEntVsDZK9wi0zrzabNeJ39GgZ8LrL-z9LN7PgKw8b5CehDgkf5dCdTygr16VUddj2M2pvJKUhNmKOz_fyBhDIBPmSOpQHTgOcjOvBoH6NkieE0Q7qdYqW22T8a7Wt2Vhvjuy436KEI18alDlnETee0azfreEET4_cA/s320/InsertDataTableDialog.png" width="320" /></a></td></tr><tr><td class="tr-caption" style="text-align: center;">Figure 2: Insert data table dialog</td></tr></tbody></table><br /><div><br /></div><p>The chart data table can be added to the chart from the menu (Insert -> Data Table...), where it is possible to select to show the data table and set the data table specific properties (see Figure 2). The data table specific properties are:</p><p></p><ul style="text-align: left;"><li>Show Horizontal Border</li><li>Show Vertical Border</li><li>Show Outline</li><li>Show Keys</li></ul><div>The properties "Show Horizontal Border" and "Show Vertical Border" control if the inner horizontal or vertical borders of the data table are shown or not. The "Show Outline" controls if the outline - borders around the data table are shown or not. "Show Key" controls if the legend keys for the data series are shown in the data table in addition to the data series (row) names.</div><div><br /></div><table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto;"><tbody><tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiaq0nrHvSIToE1oNxUB7PQsYtOnfUm14vivslIYupttaVHRpxSl-iWEAt9Id94Uv3OB5A_ZWv_XSmPpW05j0s7Q4no0FtzmHrXp2cGKkhiQg7oGEgL3nuBGzQk9E_zeZdqA086rU1DeP7yX1hsqyfeMx181TOX3rxNPdxf2klB5W8USpRm4fpwCoi1Uw/s952/DataTableDialog.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="741" data-original-width="952" height="311" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiaq0nrHvSIToE1oNxUB7PQsYtOnfUm14vivslIYupttaVHRpxSl-iWEAt9Id94Uv3OB5A_ZWv_XSmPpW05j0s7Q4no0FtzmHrXp2cGKkhiQg7oGEgL3nuBGzQk9E_zeZdqA086rU1DeP7yX1hsqyfeMx181TOX3rxNPdxf2klB5W8USpRm4fpwCoi1Uw/w400-h311/DataTableDialog.png" width="400" /></a></td></tr><tr><td class="tr-caption" style="text-align: center;">Figure 3: Data Table Dialog</td></tr></tbody></table><br /><div><br /></div><div>In addition to those data table specific properties it is also possible to change the line, fill and font properties of the data table (see Figure 3). For example the line properties define how the borders (lines) will be shown, so it is possible to change the line style (continuous line, dashes, dots), the colour of the line, transparency, line thickness. The fill properties defines the colour of the cell background and the font properties what font and size the is used for the text in the data tables.</div><div><br /></div><h2 style="text-align: left;">Implementation</h2><div>The data table implementation is located in the chart2 component, like all the other chart related code. A data table is represented in the model by a new DataTable (chart2/source/inc/DataTable.hxx) class (extending the XDataTable interface) and lives on the "Diagram" object. If there is no DataTable object, it means the data table is turned off. The DataTable holds the data table specific properties as well as line, fill and font properties.</div><div><br /></div><div>The automatic positioning of the data table is done similar like the x-axis labels, as the data table is also meant to replace the x-axis labels, unless it is not able to do so, because of the chart type (for example the "bar chart", which has the main x and y axis swapped). </div><div><br /></div><div>Rendering is done on the DataTableView class (chart2/source/view/inc/DataTableView.hxx), which creates a table shape and positions that into the chart. The row headers (data series names) and column headers (x-axis names) and values (from the data series) are filled into the table cell by cell, where also the cell properties are mapped from the model (DataTable class), so the data table looks correctly.</div><div><br /></div><h2 style="text-align: left;">Document format support</h2><div>Data table is supported by OOXML (c:dTable element). There was already present reading and writing of the basic data table properties ("horizontal border", "vertical border" and "outline" but not "keys") to preserve the document formatting even when LibreOffice couldn't render the data table itself. The properties however were not present at a convenient place (directly on the Dialog object in the model) so this had to be refactored to use a DataTable class instead. Now the OOXML support can not only preserve the complete properties (including line, fill, font) and of course also render the data table properly. </div><div><br /></div><div>Support for the ODF format had to be added into the LibreOffice extended namespace. This was done with a "data-table" element that was added to the "chart:chart" element. The "data-table" element only has a link to a certain style instance (linked with "style-name" attribute to a style:style element with the same "name" attribute). The data table specific properties are attributes of the "chart-properties" element ("loext:show-horizontal-border", "loext:show-vertical-border", loext:show-outline", "loext:show-keys" attributes), that can be added to a style. The style can also have "graphic-properties" and "text-properties", which are mapped to line, fill and text properties of the data table on import (and vice-versa on the export).</div><div><br /></div><div>The support for the data tables is currently available in LibreOffice master and will be present as a feature of LibreOffice 7.5 when released. </div><div> </div><p></p>Tomaž Vajngerlhttp://www.blogger.com/profile/17183461757847157603noreply@blogger.com0tag:blogger.com,1999:blog-2640003666891728724.post-25570429973125884712022-03-07T10:31:00.002+01:002022-03-07T10:31:18.192+01:00Sparklines in Calc<p></p>
Sparklines are mini charts available in OOXML (XLSX) documents, but until
now were not supported by LibreOffice Calc. Thanks to the funding of NGI,
this missing feature is now being implemented.
<div><br /></div>
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="float: left; margin-right: 1em; text-align: left;">
<tbody>
<tr>
<td style="text-align: center;">
<img border="0" data-original-height="541" data-original-width="811" height="48" src="https://blogger.googleusercontent.com/img/a/AVvXsEgx-whNTCTOQ9YbFy1qJ5Y275pTjYvdSFQ1sAz6lFrbthw5K3jrMoYExvpTAdsfx2Dl8SMX94LaHwBt0bUL0JNKBJY9NnNhAJX6XeOAp2zm_FhbRajJBD8sl6BvL2Y66ysjwozdcocWCyOvcpMiHmitJvz8f5hO5eRxhSbzzD4xloNfeYdYLkbRa7qn4Q=w72-h48" style="margin-left: auto; margin-right: auto;" width="72" />
</td>
<td class="tr-caption" style="text-align: center;">
<div style="text-align: left;"><br /></div><div style="text-align: left;">
This project has received funding from the European Union’s Horizon
2020 research and innovation programme under grant agreement No 871498
</div>
<div>
<div style="text-align: left;"><br /></div>
</div>
</td>
</tr>
</tbody>
</table>
<div><br /></div>
<div>
<div style="text-align: left;"><br /></div><div style="text-align: left;"><br /></div><div style="text-align: left;"><br /></div><div style="text-align: left;"><br /></div><div style="text-align: left;">To add support in LibreOffice for sparklines, we need to first read them into the LibreOffice data model, but the data model for sparklines doesn't yet exists, so we need to create that first. Sparklines are defined for one cell, but multiple sparklines can be grouped together into a group, which shares the same properties for rendering the sparkline. The unique data that is defined only for one sparkline is the data range, that a sparkline will use for rendering.</div><div style="text-align: left;"><br /></div><div style="text-align: left;">With this in mind we create a data model that consists of classes: SparklineCell -> Sparkline -> SparklineGroup, where SparklineCell is "added" to a cell and just holds a pointer to the Sparkline.</div><div style="text-align: left;"><br /></div><div style="text-align: left;">There are 3 types of sparklines supported in OOXML: line, column and stacked. The "line" type renders a line for the data range, "column" type shows each data point as a column bar, and "stacked" is a win/loss column bar, that shows if the data is positive or negative. </div><div style="text-align: left;"><br /></div><div style="text-align: left;"><br /></div><div style="text-align: left;"><table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto;"><tbody><tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/a/AVvXsEh3ElOE7YmfMWsz4bErXrxTmqucmFW_6SySFphNIz6mCh9A6K-rbQ48AHCa0BkHScQdm3W0cSCkYaAgyGx-9ipvmBtHjEthqpCe9EAsXuqLI0GPV72ML6ac1Nscm1ZB6DEg6qjl50295BOhWQ_pYNT0DQ7QGVzuBZgJu75CgnaW3UD3Thy9CgDwlQSLwg=s1698" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="1698" data-original-width="1638" height="400" src="https://blogger.googleusercontent.com/img/a/AVvXsEh3ElOE7YmfMWsz4bErXrxTmqucmFW_6SySFphNIz6mCh9A6K-rbQ48AHCa0BkHScQdm3W0cSCkYaAgyGx-9ipvmBtHjEthqpCe9EAsXuqLI0GPV72ML6ac1Nscm1ZB6DEg6qjl50295BOhWQ_pYNT0DQ7QGVzuBZgJu75CgnaW3UD3Thy9CgDwlQSLwg=w386-h400" width="386" /></a></td></tr><tr><td class="tr-caption" style="text-align: center;">Figure 1: "Line" and "Column" sparklines in LibreOffice Calc</td></tr></tbody></table><br />There are many properties for a sparklines that can be customised by the user. These are some of them:</div><div style="text-align: left;"><ul style="text-align: left;"><li>First Point, Last Point - if enabled, it shows the first and/or last point in different custom color (See Figure 1 - "column" sparkline in A3 cell, first is green and last is blue).</li><li>High Point, Low Point - if enabled, it shows the highest point and/or lowest point in different custom color (See Figure 1 - "column" sparkline at A6 cell, low points are in yellow).</li><li>Negative Point - if enabled, it shows the negative points in a different custom color (See Figure 1 - "column" sparkline at A6 cell, negative points are red).</li><li>Markes - if enabled, it shows the markers (only for "line" type) (See Figure 1 - "line" sparkline at A2 cell - shows markers).</li><li>Axis - if enabled, it shows the axis line</li><li>Right-to-left - if enabled, it shows the data in the right to left order</li><li>...</li></ul><br /><table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto;"><tbody><tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/a/AVvXsEjy2W3ySr8S2c7oiGHZGScaYqvwV9kbDCtfNHDpADf3PSThCLU0wIMysrEB6Lq6yHBYrVaT_eCEFlwB8Mj9z6dQuCxm7mNjK3Q3LGJLiB6h-tj8MCjJ8ZPR0hFCtmiPOqPUk58F5qoH6SP8Rux58dWSxqrdjbZfpmgSLVt57orCw1KgDI6Q8wNAEXNf6g=s3154" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="1104" data-original-width="3154" height="224" src="https://blogger.googleusercontent.com/img/a/AVvXsEjy2W3ySr8S2c7oiGHZGScaYqvwV9kbDCtfNHDpADf3PSThCLU0wIMysrEB6Lq6yHBYrVaT_eCEFlwB8Mj9z6dQuCxm7mNjK3Q3LGJLiB6h-tj8MCjJ8ZPR0hFCtmiPOqPUk58F5qoH6SP8Rux58dWSxqrdjbZfpmgSLVt57orCw1KgDI6Q8wNAEXNf6g=w640-h224" width="640" /></a></td></tr><tr><td class="tr-caption" style="text-align: center;">Figure 2: Horizontal and vertical sparklines of all 3 types</td></tr></tbody></table><br /><div><br /></div><div>Once we have the data model ready we can render the sparklines in the cell area. In Calc this currently looks like in Figure 1 and Figure 2. The Figure 1 shows examples of "line" and "column" sparklines (2 of each), and Figure 2 shows all three types for a block of (random) data (horizontally and vertically).</div><div><br /></div><div>Currently the code for this is in a feature branch (feature/sparklines), but is in the process of being up-streamed to master. The feature will be available in LibreOffice 7.4.</div><div><br /></div><h4 style="text-align: left;">Next step</h4><div>With the current implementation, we can open a XLSX document with sparklines and they will be rendered in LibreOffice Calc, but we can't save the document and preserve the sparklines yet. It is also not possible to create new sparklines from scratch or change any of the properties yet (there is no UI for it). ODF support is also missing.</div><div><br /></div><div>This things will be implemented in the following weeks, and when they are ready, I will blog about them again. </div><div><br /></div></div>
<p></p>
</div>
Tomaž Vajngerlhttp://www.blogger.com/profile/17183461757847157603noreply@blogger.com1tag:blogger.com,1999:blog-2640003666891728724.post-28927194676657083982021-09-13T10:43:00.001+02:002021-09-13T11:10:28.523+02:00Document searching and indexing export - Part 3<h2 style="text-align: left;">Milestone 3 - Gluing everything together into a search solution</h2><div>In the <a href="https://tomazvajngerl.blogspot.com/2021/07/document-searching-and-indexing-export.html">part 1</a> we looked into indexing XML export, and in the <a href="https://tomazvajngerl.blogspot.com/2021/08/document-searching-and-indexing-export.html">part 2</a> into rendering a search result as an image. In this part we will glue together both parts with an indexing search engine (Solr) into a full solution for searching and introduce a "proof of concept" application for searching of documents.</div><div><br /></div><div><div>Thanks to NLnet Foundation for sponsoring this work.</div></div><div><br /></div><h2 style="text-align: left;">Solr search platform</h2><div><a href="https://solr.apache.org/">Apache Solr</a> is a popular platform for searching and the idea is to use it as our search and indexing engine. First we need to figure out how to put the indexing data from our indexing XML into Solr. Solr uses the concept of documents (not to be confused with a LibreOffice document), which is an entry in the database, which can contain multiple fields. To add documents into the database, we can use a specially structured Solr XML file (many others are supported, like JSON) and simply send it using a HTTP POST request. </div><div><br /></div><div>So we need to convert our indexing XML into Solr structure, which is done like this:<br /><ul style="text-align: left;"><li>Each paragraph or object is a Solr document.</li><li>All the attributes of a paragraph or object is an field of a Solr document.<br /></li><li>The paragraph text is stored in a "content" field.</li><li>An additional field is "filename", which is the name of the source (Writer) document.</li></ul><div>For example:</div></div><div><div><span style="font-family: courier; font-size: x-small;"> <paragraph index="9" node_type="writer">Lorem ipsum</paragraph></span></div></div><div><span style="font-family: courier; font-size: x-small;"><br /></span></div><div>transforms to:</div><div><div><span style="font-family: courier; font-size: x-small;"> <add></span></div><div><span style="font-family: courier; font-size: x-small;"> <doc></span></div><div><span style="font-family: courier; font-size: x-small;"> <field name="filename">Lorem.odt</field></span></div><div><span style="font-family: courier; font-size: x-small;"> <field name="type">paragraph</field></span></div><div><span style="font-family: courier; font-size: x-small;"> <field name="index">9</field></span></div><div><span style="font-family: courier; font-size: x-small;"> <field name="node_type">writer</field></span></div><div><span style="font-family: courier; font-size: x-small;"> <field name="content">Lorem ipsum</field></span></div><div><span style="font-family: courier; font-size: x-small;"> </doc></span></div></div><div><span style="font-family: courier; font-size: x-small;"> ...</span></div><div><span style="font-family: courier; font-size: x-small;"> </add></span></div><div><br /></div><h3 style="text-align: left;">Searching using Solr</h3><div>Solr has a extensive API for querying/searching, but for our needs we just need a small subset of those. Searching is done by sending a HTTP GET to Solr server. For example with the following URL in browser:</div><div><br /></div><div><span style="font-family: courier; font-size: x-small;">http://localhost:8983/solr/documents/select?q=content:Lorem*</span></div><div><span style="font-family: courier; font-size: x-small;"><br /></span></div><div><span style="font-family: inherit;">"documents" in the URL is the name of the collection (where we put our index data), </span>"q" parameter is the query string, "content" is the field we want to search in (we put the paragraphs text in "content" field) and "Lorem*" is the expression we want to search for.</div><div><br /></div><div><br /></div><h2 style="text-align: left;">Proof of concept web application</h2><table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto;"><tbody><tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgk-Kik2F3Ao23Gr0bej_WHNkOq4k3wEzEvhFtTj9B-M7tq-1hmU8GC5KzBt9-evGX7GOqO_I3Mc_LqEYBD-XtvanVuhBwTqzXu_IVYlqLzPyPdUKT5-U3cgHvwwv0AbW8Uy3KavSeGLLJL/s1834/SearchPOC.png" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="897" data-original-width="1834" height="196" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgk-Kik2F3Ao23Gr0bej_WHNkOq4k3wEzEvhFtTj9B-M7tq-1hmU8GC5KzBt9-evGX7GOqO_I3Mc_LqEYBD-XtvanVuhBwTqzXu_IVYlqLzPyPdUKT5-U3cgHvwwv0AbW8Uy3KavSeGLLJL/w400-h196/SearchPOC.png" width="400" /></a></td></tr><tr><td class="tr-caption" style="text-align: center;">Figure 1: Search "proof of concept" web application<br /><br /></td></tr></tbody></table><div><br />The application is written in python for the server side processing and HTTP server and the client side HTML+JavaScript using <a href="https://angularjs.org/">AngularJS</a> (for data binding, REST services) and Bootstrap (UI). The purpose of the web app is to demonstrate how to implement searching and rendering in other web applications.</div><div><br /></div><div>The web app (see Figure 1) shows a list of documents in a configurable folder, where each document can be opened in Collabora Online instance. On top there is a edit filed and the "Search" button, with which we can search the documents, and a "Re-Index Documents" button, which triggers re-indexing of all the documents. </div><div><table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto;"><tbody><tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEivXuG45UN4LM5LICym8H840uQq3AVqhoya-FQNJWv4xuR0GHiGOB2s_6-fo9yQsBAumVRWjQzpfZljrJF19Ae1WsOR_u7fj7UNQVQE9Vk7rW8r-irkGdLEniLZT1bkmxR6BfkCebMYay2u/s2109/SearchPOC+-+Search+Results.png" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="1492" data-original-width="2109" height="283" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEivXuG45UN4LM5LICym8H840uQq3AVqhoya-FQNJWv4xuR0GHiGOB2s_6-fo9yQsBAumVRWjQzpfZljrJF19Ae1WsOR_u7fj7UNQVQE9Vk7rW8r-irkGdLEniLZT1bkmxR6BfkCebMYay2u/w400-h283/SearchPOC+-+Search+Results.png" width="400" /></a></td></tr><tr><td class="tr-caption" style="text-align: center;">Figure 2: Search "proof of concept" web application - Search Results<br /></td></tr></tbody></table></div><div><br /></div><div>After we enter a search expression and click the "Search" button, we get a page with search results, which is a table of the document filename and the rendered image from the document, where in the document the search result has been found. See Figure 2 for an example.</div><div>There is a "Clear" button at the bottom, which clears the search results and shows the initial list of documents again.</div><div><br /></div><h2 style="text-align: left;">About Server.py - REST and HTTP server</h2><div>The server has the following services:</div><div><ul style="text-align: left;"><li>Provide the HTML and JS documents to the browser, so the web app can be shown<br /></li><li>GET service "/document" - returns a list of documents</li><li>POST service "/search" - triggers a query in Solr and returns the result</li><li>POST service "/reindex" - triggers the re-indexing process</li><li>POST service "/image" - triggers rendering of an image for the input search result, and returns the image as base64 encoded string</li></ul><div><br /></div><h3>Re-indexing service</h3><div>Re-indexing glues together the "convert-to" service of the Collabora Online server, to get the indexing XML for a input document, conversion of the indexing XML to Solr supported XML and updating the entries in the Solr server.</div><div><br /></div><h3 style="text-align: left;">Search service</h3></div><div>Search service is using the Solr query REST service to search, and transforms the result to a JSON format, that we can use in the web app and is also compatible to use as an input to render a search result.<br /></div><div><br /></div><h3 style="text-align: left;">Image service</h3><div>Sending a search result and the document to "render-search-result" HTTP POST service on Collabora Online server, the image of the search result is rendered and sent back. For easier use in the web client, the image is converted to base64 string. </div><div><br /></div><h2 style="text-align: left;">Demo video</h2><div>Video showing searching in the WebApp:</div><div><br /></div><div class="separator" style="clear: both; text-align: center;"><iframe allowfullscreen="" class="BLOG_video_class" height="266" src="https://www.youtube.com/embed/OCYztLnCKDw" width="320" youtube-src-id="OCYztLnCKDw"></iframe></div><div><br /></div>Video showing re-indexing in the WebApp:<br /><div class="separator" style="clear: both; text-align: center;"><br /></div><div class="separator" style="clear: both; text-align: center;"><br /></div><div class="separator" style="clear: both; text-align: center;"><iframe allowfullscreen="" class="BLOG_video_class" height="266" src="https://www.youtube.com/embed/m8PTe_BChRk" width="320" youtube-src-id="m8PTe_BChRk"></iframe></div><br /><div><br /></div><h2>Proof of concept web app source location and relevant commits</h2><div style="text-align: left;">The proof of concept web application is located in Collabora Online source tree inside the indexing sub-folder. Please check the README file on how to start it up.</div><div style="text-align: left;"><br /></div><div style="text-align: left;">Collabora Online:</div><div style="text-align: left;"><div><a href="https://github.com/CollaboraOnline/online/commit/b31eb2ab92a7b9d7dbb212b07f952b8300642bdd">New POST service to render a search result + unit and integ. tests</a></div><div><a href="https://github.com/CollaboraOnline/online/commit/02c60302b3de2dce27be044d6a8d239ed786b450">Proof of concept Search WebApp to show how to implement doc. search </a></div></div><div style="text-align: left;"><div><br /></div></div><div style="text-align: left;">Fixes and changes for LibreOffice core:</div><div style="text-align: left;"><a href="https://cgit.freedesktop.org/libreoffice/core/commit/?id=50f0e8c7880122a05585a2233f6f35d0dfee0385">indexing: make indexing XML flat and use simple element names</a></div><div style="text-align: left;"><a href="https://cgit.freedesktop.org/libreoffice/core/commit/?id=0b698aa6eb26d75ca4baf677a461aee095f69317">indexing: move xml parsing into SearchResultLocator</a></div><div style="text-align: left;"><a href="https://cgit.freedesktop.org/libreoffice/core/commit/?id=f35b9909070919013a753c382bff52614403c7a2">indexing: add indexing xml as a format for writer</a></div><div style="text-align: left;"><a href="https://cgit.freedesktop.org/libreoffice/core/commit/?id=eac288d02cafc49c5a14fa27bb449c33eb4b1803">indexing: support JSON and XML as input for SearchResultLocator</a></div><div style="text-align: left;"><a href="https://cgit.freedesktop.org/libreoffice/core/commit/?id=83da70de985e1f9f7193676bef8dc71226803bb0">indexing: rename "type" for prargraph an object nodes</a></div><div style="text-align: left;"><a href="https://cgit.freedesktop.org/libreoffice/core/commit/?id=cce5a278659b1dafacdf64da8dcfab02dba25d75">indexing: rename "parent" attibute and parse the attribute back</a></div><div style="text-align: left;"><a href="https://cgit.freedesktop.org/libreoffice/core/commit/?id=160b6db92cc94f2ec2447ae0e9c60c8c9dcf3bad">indexing: fix correct size and pos. for shapes + more tests </a></div><div><br /></div><div><br /></div><div><br /></div><div><br /></div>Tomaž Vajngerlhttp://www.blogger.com/profile/17183461757847157603noreply@blogger.com0tag:blogger.com,1999:blog-2640003666891728724.post-89583231365358916602021-08-17T10:09:00.000+02:002021-08-17T10:09:17.253+02:00Document searching and indexing export - Part 2<h2 style="text-align: left;">Milestone 2 - Rendering an image of the search result</h2><div><br /></div><div>In the
<a href="https://tomazvajngerl.blogspot.com/2021/07/document-searching-and-indexing-export.html">part 1</a>, I talked about the functionality added to LibreOffice to create indexing
XML file from the document, which can be used to feed into a search indexing
engine. After we search, we expect a search hit will contain the added
internal node information from the indexing XML file. The next step is that
with the help of that information, we now render that part of the document
into an image.</div>
<div><br /></div>
<div>Thanks to NLnet Foundation for sponsoring this work.</div>
<div><br /></div>
<h3 style="text-align: left;"><table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto;">
<tbody>
<tr>
<td style="text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgmpDZeRUfkdW3xq4Ota8Hw9yDQi9ggyQAArnzYUuZDD6bHgXDF5y5QgoRKaTR4hxrR2vlhULbm93svhpdeT38zejzZOgs5dcfMsUr508JHHIarNF5jv2FlflXpm4kwJ-DEpAsfOUQClqap/s946/Search+Rectangles.png" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="591" data-original-width="946" height="400" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgmpDZeRUfkdW3xq4Ota8Hw9yDQi9ggyQAArnzYUuZDD6bHgXDF5y5QgoRKaTR4hxrR2vlhULbm93svhpdeT38zejzZOgs5dcfMsUr508JHHIarNF5jv2FlflXpm4kwJ-DEpAsfOUQClqap/w640-h400/Search+Rectangles.png" width="640" /></a>
</td>
</tr>
<tr>
<td class="tr-caption" style="text-align: center;">
Figure 1: Example of a rectangle for a search string
</td>
</tr>
</tbody>
</table><br /></h3><h3 style="text-align: left;">Calculating the result rectangle</h3>
<div>
To render an image, we first need to get the area of the document, where the
search hit is located. This is implemented in the <span style="font-family: courier;">SearchResultLocator</span><span style="font-family: inherit;"> class, which takes </span><span style="font-family: courier;">SearchIndexData</span><span style="font-family: inherit;"> that contains the internal model index of the hit location (object
and paragraph). The algorithm then finds the location of the paragraph in
the document model, and then it determines, what the rectangle of the
paragraph is.</span>
</div>
<div>
<span style="font-family: inherit;"><br /></span>
</div>
<div>
<span style="font-family: inherit;">The search hit can span over multiple paragraphs, so we need to handle
multiple hit locations. With that we get multiple rectangles, which need to
be combine into the final rectangle (union of all rectangles). See figure 1
for an example.</span>
</div>
<div>
<span style="font-family: inherit;"><br /></span>
</div>
<h3 style="text-align: left;">
Rendering the image from the rectangle in LOKit<br />
</h3>
<div>
This part is implemented for the LOKit API, which can already handle rendering
part of the document with an existing API, using rendering of the tiles.
</div>
<div><br />The new function added to the API is:</div>
<div>
<div class="add">
<span style="font-family: courier;">bool renderSearchResult(const char* pSearchResult, unsigned char**
pBitmapBuffer, </span><span style="font-family: courier;">int* pWidth, int* pHeight, size_t* pByteSize);</span>
</div>
</div>
<div class="add">
<span style="font-family: courier;"><br /></span>
</div>
<div class="add">
<span style="font-family: inherit;">The method renders an image for the search result. The input is the </span><span style="font-family: courier;">pSearchResult</span><span style="font-family: inherit;"> (XML), and </span><span style="font-family: courier;">pBitmapBuffer</span>, <span style="font-family: courier;">pWidth</span>, <span style="font-family: courier;">pHeight</span><span style="font-family: inherit;">, </span><span style="font-family: courier;">pByteSize</span><span style="font-family: inherit;"> are output parameters</span><span style="font-family: courier;">.</span>
</div>
<div class="add">
<span style="font-family: courier;"><br /></span>
</div>
<div class="add">
<span style="font-family: inherit;">If the command </span>succeeded, the
function returns <span style="font-family: courier;">true</span><span style="font-family: inherit;">, the </span><span style="font-family: courier;">pBitmapBuffer</span><span style="font-family: inherit;"> contains the raw image, </span><span style="font-family: courier;">pWidth</span><span style="font-family: inherit;"> and </span><span style="font-family: courier;">pHeight</span><span style="font-family: inherit;">
contain the width and height of the image in pixels, and </span><span style="font-family: courier;">pByteSize</span><span style="font-family: inherit;"> the byte size of the image. </span>
</div>
<div class="add">
<span style="font-family: inherit;"><br /></span>
</div>
<div class="add">
<span style="font-family: inherit;">What happens internally in the function is, that the content of </span><span style="font-family: courier;">pSearchResult</span><span style="font-family: inherit;">
is parsed with a XML parser, so that a </span><span style="font-family: courier;">SearchIndexData</span> can be
created and send to <span style="font-family: courier;">SearchResultLocator</span> to get the rectangle of the search hit area. A call to <span style="font-family: courier;">doc_paintTile</span> then renders the part of the document enclosed by the rectangle to the
input <span style="font-family: courier;">pBitmapBuffer</span>.
</div>
<div class="add"><br /></div>
<div class="add">
See <a href="https://cgit.freedesktop.org/libreoffice/core/tree/desktop/source/lib/init.cxx?id=e1511ce551f27a5560600029193f076fd65ece17#n5736">desktop/source/lib/init.cxx</a> - function "<span style="font-family: courier;">doc_renderSearchResult</span>"
</div>
<div class="add"><br /></div>
<h3 style="text-align: left;">
Collabora Online service "render-search-result"<br />
</h3>
<div>
To actually be useful, we need to provide the functionality in a form that can
be "glued" together with the search provider and indexer to show the rendered
image of the search hit from the document. For this we have implemented a
service in the Collabora Online. The service is a just a HTTP POST
request/response, where the in the request we send the document and the search
result to the service, and the response is the image.
</div>
<div><br /></div>
<div>What the service does is:</div>
<div>
<ul style="text-align: left;">
<li>load the document</li>
<li>run the "renderSearchResult" with the search result XML</li>
<li>interpret the bitmap and encode into the PNG format</li>
<li>return the PNG image</li>
</ul>
<div>
As an example how the service can be used, see in Collabora Online
repository: test/integration-http-server.cpp - test method
HTTPServerTest::testRenderSearchResult
</div>
</div>
<div>
<span style="font-family: inherit;"><br /></span>
</div>
<div>
<div>
The following commits are implementing this milestone 2 functionality:
</div>
</div>
<div><br /></div>
<div>Core:</div>
<div>
<a href="https://cgit.freedesktop.org/libreoffice/core/commit/?id=6e155959de6f46afbe0b68057200c3da822d81f9">indexing: search result locator to return the rect of the result</a>
</div>
<div>
<div class="commit-subject">
<a href="https://cgit.freedesktop.org/libreoffice/core/commit/?id=e1511ce551f27a5560600029193f076fd65ece17">indexing: add LOKit API to render the search result into a bitmap</a>
</div>
</div>
<div class="commit-subject">
<div class="commit-subject">
<a href="https://cgit.freedesktop.org/libreoffice/core/commit/?id=ea1818b8ba34378b777b8706069d28fade2cc924">indexing: add support for SdrObjects in SearchResultLocator</a>
</div>
<div class="commit-subject">
<div class="commit-subject">
<a href="https://cgit.freedesktop.org/libreoffice/core/commit/?id=df3d733724de84a2b54398434b621049a326c4d8">indexing: add "type" to the xml for paragraph nodes</a>
</div>
<div class="commit-subject">
<div class="commit-subject">
<a href="https://cgit.freedesktop.org/libreoffice/core/commit/?id=7da5537f6a43c1b82afc5e0c8d18b8d847293fda">indexing: use XML as input that is identical to indexing XML</a>
</div>
<div class="commit-subject">
<a href="https://cgit.freedesktop.org/libreoffice/core/commit/?id=adf65471e889676a600a9c6d0454c75cbd549ad3">indexing: allow for multiple entries in search indexing data</a>
</div>
<div class="commit-subject"><br /></div>
<div class="commit-subject">Online:</div>
<div class="commit-subject">
<a href="https://github.com/quikee/online/commit/d79dca011f32623c39eb721b1b1adaef06c2b170">
New POST service to render a search result + unit and integ. tests
</a>
</div>
<div class="commit-subject"><br /></div>
<div class="commit-subject">
<div>
In the next milestone, we will glue everything together into a
searching solution.
</div>
<div><br /></div>
</div>
</div>
</div>
</div>
<div><span style="font-family: inherit;"> </span></div>
Tomaž Vajngerlhttp://www.blogger.com/profile/17183461757847157603noreply@blogger.com0tag:blogger.com,1999:blog-2640003666891728724.post-63668428679112665922021-07-01T15:12:00.000+02:002021-07-01T15:12:05.058+02:00Document searching and indexing export - Part 1<h2 style="text-align: left;">About the idea</h2><p>Searching for a phrase in multiple documents is not a new thing and many
implementations exist, however such searching will usually only provide you if
and roughly where in the documents a searched phrase exists. With Collabora
Online and LibreOffice we can do better than this and in addition provide the
search result in form of a thumbnail of the search location. In this way it is
easier for the user to see the context, where the searched phrase is located.
For example, if it is located in a table, shape, footer/header, or is it
figure text or maybe "alt" text of an image.
</p>
<p>
Thanks to the sponsor of the work - NLnet Foundation, we are implementing this
solution for Writer documents.
</p>
<p>The solution to this consist in 3 parts: <br /></p>
<ul style="text-align: left;">
<li>preparing the data for indexing, </li>
<li>indexing and searching </li>
<li>rendering of the result</li>
</ul>
Preparing the data for indexing and rendering of the search result is done in
LibreOffice core, while the actual indexing and searching is delegated to one of
the existing indexing and searching databases / frameworks (we will provide
support for Apache Solr).
<p></p>
<p>In this post I will describe what has been done for milestone 1.</p>
<h2 style="text-align: left;">Milestone 1 - preparing data for indexing</h2>
<div>
Indexing data usually consists of (enriched) text, however in our case we also
need to provide additional internal information, where the text is located, so
it is possible to later go to the search result location and create a
thumbnail of the document. In Writer we can provide a node index of the
paragraph, with which it is possible to quickly identify the text in the
document model and generate a thumbnail of the area around the text.</div><div><br /></div>
<div>
The data for indexing is provided by a "indexing export" filter in
LibreOffice, which creates a XML document with a custom structure. The root
element is <span style="font-family: courier;"><indexing></span><span style="font-family: inherit;"> and the child elements are paragraphs with index and text, which can
be nested in sub-elements (like image, shape, table, section) depending on
where the paragraph is located. </span>
</div>
<div>
<span style="font-family: inherit;"><br /></span>
</div>
<div><span style="font-family: inherit;">For example:</span></div><div><span style="font-family: inherit;"><br /></span></div>
<div>
<span style="font-size: x-small;"><span style="font-family: inherit;"> </span><span style="font-family: courier;"><?xml version="1.0" encoding="UTF-8"?></span>
</span></div>
<div><span style="font-family: courier; font-size: x-small;"><indexing></span></div>
<div>
<span style="font-size: x-small;"><span style="font-family: courier;"> <paragraph index="6">Drawing : Just a
Diamond</paragraph></span>
</span></div>
<div>
<span style="font-size: x-small;"><span style="font-family: courier;"> <paragraph index="12"></paragraph></span>
</span></div>
<div>
<span style="font-size: x-small;"><span style="font-family: courier;"> <shape name="Circle" alt="" description=""></span>
</span></div>
<div>
<span style="font-size: x-small;"><span style="font-family: courier;"> <paragraph index="0">This is a circle</paragraph></span>
</span></div>
<div>
<span style="font-size: x-small;"><span style="font-family: courier;"> <paragraph index="1">This is a second
paragraph</paragraph></span>
</span></div>
<div><span style="font-family: courier; font-size: x-small;"> </shape></span></div>
<div>
<span style="font-size: x-small;"><span style="font-family: courier;"> <shape name="Diamond" alt="" description=""></span>
</span></div>
<div>
<span style="font-size: x-small;"><span style="font-family: courier;"> <paragraph index="0">This is a diamond</paragraph></span>
</span></div>
<div><span style="font-family: courier; font-size: x-small;"> </shape></span></div>
<div>
<span style="font-size: x-small;"><span style="font-family: courier;"> <shape name="Text Frame 1" alt="" description=""></span>
</span></div>
<div>
<span style="font-size: x-small;"><span style="font-family: courier;"> <paragraph index="0">This is a TextBox -
Para1</paragraph></span>
</span></div>
<div>
<span style="font-size: x-small;"><span style="font-family: courier;"> <paragraph index="1">Para2</paragraph></span>
</span></div>
<div>
<span style="font-size: x-small;"><span style="font-family: courier;"> <paragraph index="2">Para3</paragraph></span>
</span></div>
<div><span style="font-family: courier; font-size: x-small;"> </shape></span></div>
<div><span style="font-family: courier; font-size: x-small;"></indexing></span></div>
<div><br /></div>
<div>
The indexing export is build upon a ModelTraverser class, which was created
for the indexing purpose, but can be reused for other purposes (it is similar
to what AccessibilityCheck does, but generalised, so AccessibilityCheck can in
the future be refactored to use it).
</div>
<div><br /></div>
<div>
The purpose of ModelTraverser is to traverse through the Writer document
model, and provide SwNode and SdrObjects to the consuming objects - in our
case IndexingExport class, which extracts the text from those objects
(depending on the object type) and with help of a XmlWriter, writes the
indexing data to the XML file.
</div>
<div><br /></div><div>Indexing export filter can be tested with the LibreOffice command line "convert-to" tool in the following way:</div><div><span style="font-family: courier;"><br /></span></div><div><span style="font-family: courier; font-size: x-small;">soffice --convert-to xml:writer_indexing_export <Writer document file path></span></div><div><span style="font-family: courier;"><br /></span></div><div><span style="font-family: courier;"><br /></span></div>
<div>The commits implementing this milestone 1 functionality:</div>
<div>
<ul style="text-align: left;">
<li>
<a href="https://cgit.freedesktop.org/libreoffice/core/commit/?id=686c35a3c37674d08c1f8585a621143c959b3f29" target="_blank">indexing: indexing paragraph text with the ModelTraverser</a>
</li>
<li>
<a href="https://cgit.freedesktop.org/libreoffice/core/commit/?id=f81115740bac985cad3cd15348f75c2c78b8843a" target="_blank">indexing: indexing graphics for the IndexingExport</a>
</li>
<li>
<a href="https://cgit.freedesktop.org/libreoffice/core/commit/?id=37cd5bd4fdf2317709882c933844c4cc67e4cee4" target="_blank">indexing: indexing OLE objects for the IndexingExport</a>
</li><li><a href="https://cgit.freedesktop.org/libreoffice/core/commit/?id=c23408aeafc1f2ce3f7dde685d0b519d8e2daff7" target="_blank">indexing: indexing shapes/text boxes for the IndexingExport</a></li><li><a href="https://cgit.freedesktop.org/libreoffice/core/commit/?id=b3a62ab0a43510cf43c88aa4d6e145e46db7e7e5" target="_blank">indexing: indexing tables for the IndexingExport</a></li><li><a href="https://cgit.freedesktop.org/libreoffice/core/commit/?id=705fe1c44b41cd11518069e0627d0f48a65a7dfc" target="_blank">indexing: write parent index to paragraphs if possible</a></li><li><a href="https://cgit.freedesktop.org/libreoffice/core/commit/?id=2ce10eedfb1de8beef8ddfa457f1e19545846f86" target="_blank">indexing: indexing sections for the IndexingExport</a></li><li><a href="https://cgit.freedesktop.org/libreoffice/core/commit/?id=922e2610a4745c6e33d9e8e808131922a3ea0dc4" target="_blank">indexing: add test case for fontworks and footer/header paragraphs</a></li><li><a href="https://cgit.freedesktop.org/libreoffice/core/commit/?id=95b06d3aa514ce83f82fd538d1731fc6363e4b8a" target="_blank">indexing: add indexing export as an export filter for Writer</a></li>
</ul>
</div>
<div><br /></div>
<div>In the next milestone, we will render the thumbnail with the provided search result data.</div><div><br /></div><div><br /></div><div>To be continued...</div><div><br /></div>
Tomaž Vajngerlhttp://www.blogger.com/profile/17183461757847157603noreply@blogger.com0tag:blogger.com,1999:blog-2640003666891728724.post-2500659029599801852021-05-13T16:13:00.000+02:002021-05-13T16:14:34.331+02:00Command Popup HUD for LibreOffice<div style="text-align: left;">Command Popup is a pop-up window that lets you search for commands that are present in the main menu and run them. This was requested in bug <a href="https://bugs.documentfoundation.org/show_bug.cgi?id=91874">tdf#91874</a> and over-time accumulated over 14 duplicated bugs reports, so it was a very requested feature.</div><div style="text-align: left;"><br /></div><div style="text-align: left;">I'm intrigued by similar functionality in other programs, because it enables very quick access to commands (or programs) and at the same time don't need to move your hand off the keyboard. It also makes it easy to search for commands - especially in an application like LibreOffice with humongous main menu. So I decided to try to implement it for LibreOffice.</div><p></p><table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto;"><tbody><tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhosTK27zbFPAYtvamSZ6_hTqJJmwQspQWZ1JSSe_rdKy2FOotLSVp7y6_ZXDYERzdM8ry0HXlvS83UBWHbrBWD1FKNP3fu3M1xne0jpTPlCkHb1xR7Vkfu5BWSYrZyYg3Ws5VZtB1qpAth/s2694/CommandPopup.png" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="1659" data-original-width="2694" height="394" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhosTK27zbFPAYtvamSZ6_hTqJJmwQspQWZ1JSSe_rdKy2FOotLSVp7y6_ZXDYERzdM8ry0HXlvS83UBWHbrBWD1FKNP3fu3M1xne0jpTPlCkHb1xR7Vkfu5BWSYrZyYg3Ws5VZtB1qpAth/w640-h394/CommandPopup.png" width="640" /></a></td></tr><tr><td class="tr-caption" style="text-align: center;">Figure 1: Command Popup window<br /><br /></td></tr></tbody></table>I was working on it here and there in my free time and managed to make it work as I imagined, however it was very rough around the edges and needed a lot of polish. Luckily in April, we had a hack week at <a href="https://www.collaboraoffice.com/">Collabora</a>, where I decided to use some time to work on finishing the command popup. I dusted up the old code and converted it to use the weld framework for widgets and fixed the many bugs, but I didn't manage to finish it completely so it took until recently that I actually pushed the code upstream into <a href="https://cgit.freedesktop.org/libreoffice/core/commit/?id=83d91ebbbda2204af9a09a921055a850a16911e0">master</a>.<p></p><div style="text-align: left;">The main UX focus is to easy search and navigate with the keyboard. When the Command pop-up is focused, all keyboard events should go to the search edit box, so it is possible to change the search term, however hitting up/down should change the selection in the tree view, where the search results are shown, and enter should execute the command. To get this working correctly was quite a challenge, but I found the correct formula eventually after trying some different ideas. Of course using the mouse should still work as well. </div><div style="text-align: left;"><br /></div><div style="text-align: left;">To show the Command Popup, there is a menu entry in "Help > Search Commands" and is by default bind to "Ctrl+F1" shortcut (however this may change). </div><div style="text-align: left;"><br /></div><div style="text-align: left;">The Command Popup will be available in LibreOffice 7.2, but if you want to try it out, you can get the current daily build, or wait for the LibreOffice 7.2 Alpha1. Any suggestions and comments are welcome. If you find a bug, please report it in the <a href="https://bugs.documentfoundation.org/">LibreOffice bugzilla</a> page.</div><div style="text-align: left;"><br /></div><div style="text-align: left;"><br /></div>Tomaž Vajngerlhttp://www.blogger.com/profile/17183461757847157603noreply@blogger.com3tag:blogger.com,1999:blog-2640003666891728724.post-80566727015383349792021-03-24T08:42:00.000+01:002021-03-24T08:42:04.871+01:00Built-in "Xray" like UNO object inspector – Part 3<p>
DevTools implementation has been completed and this is the third and final
part of the mini-series. The focus of this part is on the object
inspector, but I have also improved or changed other DevTools parts, so first
I will briefly mention those.
</p>
<h1 style="text-align: left;">New menu position</h1><div><br /></div>
<div><table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto;"><tbody><tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgbea3m2bTuIQfJZO9dgAYsi9hewpA8gHy2JYqZjgItJK5QYTQ2BM6qh4rITEFT6HFVffM9KUDJOgGET3r8-2hFXktrt9VLZzLMvpFU0fWxJZvclY12pkxfP0B80TZQ8vy7iQsW_vd45jJ4/s1513/Menu.png" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="1513" data-original-width="1186" height="320" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgbea3m2bTuIQfJZO9dgAYsi9hewpA8gHy2JYqZjgItJK5QYTQ2BM6qh4rITEFT6HFVffM9KUDJOgGET3r8-2hFXktrt9VLZzLMvpFU0fWxJZvclY12pkxfP0B80TZQ8vy7iQsW_vd45jJ4/s320/Menu.png" /></a></td></tr><tr><td class="tr-caption" style="text-align: center;">Figure 1: Development Tools location in the menu</td></tr></tbody></table><br /></div><div>
The position in menu has changed from "Help / Development Tools" to
"Tools / Development Tools". The new position fits better as it is near the
position of macro editor and they go hand in hand with each-other.<br />
</div>
<h1 style="text-align: left;">Updates to the document model tree view</h1>
<div><div><br /></div><div><br /><table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto;"><tbody><tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEglbVYAw6lS1jd9utZ70OMv7hHWXFQAnkAUL0Vcdn2VePrHfxrtOe9yg1AIFskQ9wKET9FvItG5PpTMD8JltbZ_XTw5IVA8tUU4aMjYimc-sUXMzyZJxnUvZlPY0gEpKpMd-4K9dqG-Xctw/s752/Menu.png" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="752" data-original-width="511" height="400" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEglbVYAw6lS1jd9utZ70OMv7hHWXFQAnkAUL0Vcdn2VePrHfxrtOe9yg1AIFskQ9wKET9FvItG5PpTMD8JltbZ_XTw5IVA8tUU4aMjYimc-sUXMzyZJxnUvZlPY0gEpKpMd-4K9dqG-Xctw/w271-h400/Menu.png" width="271" /></a></td></tr><tr><td class="tr-caption" style="text-align: center;">Figure 2: Document model tree view and "Current Selection" toggle button<br /></td></tr></tbody></table></div><div><br /></div><div>
The left-hand side document model tree view has been changed to use the same
approach than the object inspector tree view uses, where the object attached
tree view (in this case <span style="font-family: courier;">DocumentModelTreeEntry</span>) has all the behavioural logic
for a specific tree view node. This makes it easier to manipulate the
tree view and add new nodes (if necessary).
</div><div><br /></div>
<div>
In the document object tree view I have added a top toolbar with "Refresh" (icon) button and I changed the existing “Current Selection” button to a toolbar
toggle button, so it looks more consistent as the is only the toolbar now (see figure 2).
</div><div><br /></div>
<div>
In Writer, each paragraph tree view node now has text portion child nodes (as shown in figure 2), to make it possible to quickly inspect all the individual parts of a paragraph.</div></div>
<h1 style="text-align: left;">The object inspector tree view</h1>
<div>
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto;"><tbody><tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg9HMEwu1luLOMXzYO-6ey0_TzDbsVHMmg9fdp9u_J3e8O630DkaxknSh3KM_5iwGi1jJHavjWNtQ4OJl0FZBrpupoCjZ0Z8A3yd4xbRpM65nTunMu1ova7asdvaVhv0ZwIfR3mEkx0LqpX/s1442/ObjectInspectorTabBar.png" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="229" data-original-width="1442" height="102" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg9HMEwu1luLOMXzYO-6ey0_TzDbsVHMmg9fdp9u_J3e8O630DkaxknSh3KM_5iwGi1jJHavjWNtQ4OJl0FZBrpupoCjZ0Z8A3yd4xbRpM65nTunMu1ova7asdvaVhv0ZwIfR3mEkx0LqpX/w640-h102/ObjectInspectorTabBar.png" width="640" /></a></td></tr><tr><td class="tr-caption" style="text-align: center;">Figure 3: Top toolbar and tab bar in object inspector</td></tr></tbody></table><br /><div><br /></div><div>
The object inspector shows various information about an object. Each object
has a implementation class, which is always shown for the current object (see figure 3).
</div><div><br /></div>
<div>The other information that the object inspector shows are divided into four main categories:</div>
<div><ul style="text-align: left;"><li>"Interfaces" - the interfaces that the current object implements</li><li>"Services" - the services that the current object supports</li><li>"Properties" - the properties of the current object</li><li>"Methods" - the combined methods that can be called on the current object</li></ul></div>
<div>
On the user interface, the categories are divided with a tab bar (see figure 3). Each tab represents a different categories. The tabs are filled on "entry" - when the user clicks on the tab.</div><div><br /></div>
<div>In the code the tabs and tree views are all handled by the <span style="font-family: courier;">ObjectInspectorTreeHandler</span>
and the hierarchy of objects attached the tree view that implement the
<span style="font-family: courier;">ObjectInspectorNodeInterface</span> (see
<span style="font-family: courier;">include/sfx2/devtools/ObjectInspectorTreeHandler.hxx</span>).
</div>
<div><br /></div><div>The two major categories are "Properties" and "Methods", which I describe in more detail next.</div>
</div>
<h2 style="text-align: left;">Properties</h2>
<div>
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto;"><tbody><tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg8NcqvHOP8Xj-55fjEOjc67d28_20JBO0DckXI1O5N3777NRstAy7xd8qexWL-_nZfNuJLXIHgGYxKVKBoPFSdQoOsXaDMgzkLIHr544I2wUtz-G20xNLby2DoMzYWkkGV84T20eMlqfmL/s3170/Properties.png" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="1168" data-original-width="3170" height="236" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg8NcqvHOP8Xj-55fjEOjc67d28_20JBO0DckXI1O5N3777NRstAy7xd8qexWL-_nZfNuJLXIHgGYxKVKBoPFSdQoOsXaDMgzkLIHr544I2wUtz-G20xNLby2DoMzYWkkGV84T20eMlqfmL/w640-h236/Properties.png" width="640" /></a></td></tr><tr><td class="tr-caption" style="text-align: center;">Figure 4: Object inspector "Properties" tab</td></tr></tbody></table><br /><div><br /></div>
<div>There are three types of properties of an object:</div><div><ul style="text-align: left;"><li>Properties accessible view XPropertySet.</li><li>Properties defined as an attribute (marked "[attribute]" in IDL). </li><li>Pseudo properties defined by a get and set method. </li></ul></div>
<div>
All the three types are represented in the properties tab and can be identified
by the "Info" column. Attribute properties have an "attribute" flag, pseudo
properties have either a "get", "set" or both flags. If neither flags exists,
it represents a property that is from XPropertySet. </div><div><br /></div>
<div>
In the tree view there are four columns for each property - "Name", "Value",
"Type" and already mentioned "Info". The "Name" of the property is always
available, but it is possible that two properties have the same name (because
of multiple property types). The "Value" shows the value of the property,
which is converted to the string, if this is possible (it should be if the
property is basic), otherwise a representation string is created. The representation string for objects shows the implementation name ("<span style="font-family: courier;"><Object@SwXTextRange></span>"), for
sequences it shows the size ("<span style="font-family: courier;"><Sequence [5]></span>") and for structs it just mentions the type ("<span style="font-family: courier;"><Struct></span>").
</div><div><br /></div>
<div>
A node in the “Properties” tree view can be expanded (if offered) so it is
possible to recursively inspect the the objects further. In case the property is a
struct, it shows the struct members, and if it is a sequence, it shows the
indices each object has in the sequence.</div>
<div><br /></div><div>
There are special properties, which names start with "<span style="font-family: courier;">@</span>". This are added for
convenience, so it is possible to inspect objects, that implement
XIndexContainer, XNameContainer or XEnumeration interfaces. When such an
object is found, the entries gathered using those interfaces are added to
the tree view, and are prefixed with "<span style="font-family: courier;">@</span>". For example "<span style="font-family: courier;">@3</span>"
(XIndexContainer) "<span style="font-family: courier;">@PageStyles</span>" (XNameContainer or
XEnumeration). The type of the special property is written in the "Info" column ("index
container", "name container" and "enumeration" flags). Note that this functionality is not present in Xray or the Macro editor's debugger, but was added for convenience.</div><div><br /></div>
<div><table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto;"><tbody><tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjQEV5zFwVPe9SoWa-eL1bo1oTLqan97hKhyphenhyphenuj2RFcoyFKca0RaIAftmU5HEMOJOO2edcN8NYfFSaTPBGGJoB8xGEc62NDmRQjgFLKtHI9HMRVS6rP30nbi-_Tz4jzxthFzpQYB3r9iXoNd/s2480/Properties_TextView.png" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="1179" data-original-width="2480" height="304" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjQEV5zFwVPe9SoWa-eL1bo1oTLqan97hKhyphenhyphenuj2RFcoyFKca0RaIAftmU5HEMOJOO2edcN8NYfFSaTPBGGJoB8xGEc62NDmRQjgFLKtHI9HMRVS6rP30nbi-_Tz4jzxthFzpQYB3r9iXoNd/w640-h304/Properties_TextView.png" width="640" /></a></td></tr><tr><td class="tr-caption" style="text-align: center;">Figure 5: "Properties" tab and the text view<br /><br /></td></tr></tbody></table>On the bottom of the "Properties" tab there is a text view, which shows the full value of the current selected property. In the tree view, the value shown is always using a short form (shortened to 60 chars with new-line characters removed) to not make the tree view too wide. The full value therefor is written in the text view, where it can be inspected in full and has also a working copy/paste (see figure 5).</div><h3 style="text-align: left;">Object Stack</h3><div>Related to the "Properties" is the object stack. It is possible to select an object in the tree view and inspect the object (either using the context menu or the toolbar "Inspect" action). In this case the object in the object inspector will change to the selected one. This is convenient when you are only interested in one object down in the tree view hierarchy and want to inspect only that. In that case the previous object will be added to the stack, and can be returned to with hitting the "Back" button in the toolbar.</div><div><br /></div><div>Note that going to another object (not using "Inspect" action) will always remove the object stack. </div>
</div>
<h2 style="text-align: left;">Methods</h2><div><div><table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto;"><tbody><tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgHD1J5xehfPh6-qsWhPKAbSyKVtTKTP8TiwmVf7TEC46XswC53VMsEPFAjGztOrFMBgq2QiD4w125TwkvfvIrovBGJQ7PkFIsN-xJNLLHfHU0Qcfb0gZ53BDi5Ap7lWy7OoPvWQQ83R209/s2823/Methods.png" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="1010" data-original-width="2823" height="228" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgHD1J5xehfPh6-qsWhPKAbSyKVtTKTP8TiwmVf7TEC46XswC53VMsEPFAjGztOrFMBgq2QiD4w125TwkvfvIrovBGJQ7PkFIsN-xJNLLHfHU0Qcfb0gZ53BDi5Ap7lWy7OoPvWQQ83R209/w640-h228/Methods.png" width="640" /></a></td></tr><tr><td class="tr-caption" style="text-align: center;">Figure 6: Object inspector "Methods" tab</td></tr></tbody></table><br /></div><div><br /></div>
<div>
The "Methods" tab contains a tree view that shows all the methods, that can be called for the
current object. Each method is represented by four columns (see figure 6):</div><div><ul style="text-align: left;"><li>"Method" - name of the method</li><li>"Return type" - the return (simplified) type of the method</li><li>"Parameters" - list of input parameters, where each one lists the direction ("in", "out" or "in/out"), the parameter name and the simplified type </li><li>"Implementation Class" - class/interface where the method is implemented</li></ul><div>Currently the types of parameters and return types are simplified, with only basic types, "void", "any", "sequence" and "object" that represents all the objects and the type of the object isn't written. The reason for this is to make it easier to read. </div><div><br /></div><h1 style="text-align: left;">Future ideas</h1></div><div>There are many improvements that can still be made, but aren't included in the current implementation. </div><div><br /></div><div>I think it would be quite convenient to have the ability to open a object inspector in mode-less dialog separate to the DevTools, just to quickly look up a property. </div><div><br /></div><div>Another big upgrade would also be the ability to change values of basic types for the properties and structs, so it is possible to quickly see what effect the change would have. Similar to changing property values is to call methods with defining the parameters, but only if the parameters are basic types.</div><div><br /></div><div>My initial vision of DevTools was not that it will be only one tool (object inspector), but more tools just like the development tools in the browser, so I'm sure there will be more useful things integrated over time.</div><div><br /></div><div>I think there are a lot of ideas you may also have, so please tell me if you have a good one. Of course if you find something that is not working as expected, please let me know.</div><div><br /></div>
</div>
<h1 style="text-align: left;">Credits</h1>
<div>
Many thanks to <a href="https://www.documentfoundation.org/">TDF</a> and users that support the foundation by providing
donations, to make this work possible.
</div>
<div><br /></div>
Tomaž Vajngerlhttp://www.blogger.com/profile/17183461757847157603noreply@blogger.com6tag:blogger.com,1999:blog-2640003666891728724.post-34611380299853140962021-03-01T13:49:00.000+01:002021-03-01T13:49:11.433+01:00Built-in "Xray" like UNO object inspector – Part 2<p>Since my last blog post I've been continuing the work on DevTools and since then a lot of things have progressed. Point & click has been implemented and the object inspector view has been greatly improved to show current object’s properties and methods. In this part I will mainly talk about the point & click and a bit about the current state, and in the next blog I will extensively talk about the object inspector.</p><h1 style="text-align: left;">Point & click</h1><table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto;"><tbody><tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjql87UJ7yfyeQcE8V5CA0wNnzdK1pf9oWLIVSXPbcPXxbaKCM0ZUGqeuEnQt17lN2YEjh29nfwVLn4-jkiavkr3nD5iSLhxZV9m9k2TQ11FaT7V_SjVzs6qRK4dQ3o7TQUBsg3vHn_qR_1/s1092/CurrentSelectionButton.png" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="602" data-original-width="1092" height="220" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjql87UJ7yfyeQcE8V5CA0wNnzdK1pf9oWLIVSXPbcPXxbaKCM0ZUGqeuEnQt17lN2YEjh29nfwVLn4-jkiavkr3nD5iSLhxZV9m9k2TQ11FaT7V_SjVzs6qRK4dQ3o7TQUBsg3vHn_qR_1/w400-h220/CurrentSelectionButton.png" width="400" /></a></td></tr><tr><td class="tr-caption" style="text-align: center;">Figure 1: Current selection button</td></tr></tbody></table><p><br />The idea of this functionality is to provide a way to inspect selected objects in the document, for example an image or a shape. For this, I have implemented a selection change listener (<span style="font-family: courier;">sfx2/source/devtools/SelectionChangeHandler.hxx</span>), whose purpose is to listen to the selection changes that happen in the document and store the latest selection object. It is started when the DevTools docking window is instantiated and shown. I have added a new toggle button “Current Selection” (see Figure 1) to the UI. When the button is selected, it automatically shows the current selected object (gathered with the selection change listener) in the object inspector. </p><table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto;"><tbody><tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjsNo9o12gC7zWfScdq4WcSpeZsy7tGekwc9gW0_v4GgKT0PK21XL-RmDc0wTCoHQrJSa0C0DYn_HOhVSDw4mZQm3Or-3gV7lf3TOvQM33d9XRHXP_Qf1NnXr_AgorHmUifBpI_3-L5x6k6/s2240/CurrentSelection.png" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="1954" data-original-width="2240" height="349" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjsNo9o12gC7zWfScdq4WcSpeZsy7tGekwc9gW0_v4GgKT0PK21XL-RmDc0wTCoHQrJSa0C0DYn_HOhVSDw4mZQm3Or-3gV7lf3TOvQM33d9XRHXP_Qf1NnXr_AgorHmUifBpI_3-L5x6k6/w400-h349/CurrentSelection.png" width="400" /></a></td></tr><tr><td class="tr-caption" style="text-align: center;">Figure 2: Current selected shape's properties shown in the object inspector</td></tr></tbody></table><br /><p>In the example shown in Figure 2, we can see the shape is selected in the document and its properties are shown in the object inspector. If the "Current Selection" button wouldn't be toggled, then the document top-level object would be shown in the object inspector or the selected object in the DOM tree view.</p><p>While the "Current Selection" button is toggled, selecting any object in the DOM tree view (left-hand side tree view) has no effect, however if the current selected object is also present in the current DOM tree view, it will be selected. Note that if the object is not present in the tree, it won't be selected, because the DOM tree view will not force creation of on-demand object because of performance considerations.</p><table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto;"><tbody><tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgwCCxaCUgO8ROIdnRBQPn1T_RiisXrT6tWWrc72-kdpikFAW4eCWycfFdDZIUQ-1nMOSc8R1dVjLLvZQtfhmFGQmc3UQAec7mBbYa3WafIlb0MJLEvRFJOPHLmuBRUcISWt0pHzGmBxNT_/s1804/InspectObjectInCustomizeDialog.png" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="1728" data-original-width="1804" height="384" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgwCCxaCUgO8ROIdnRBQPn1T_RiisXrT6tWWrc72-kdpikFAW4eCWycfFdDZIUQ-1nMOSc8R1dVjLLvZQtfhmFGQmc3UQAec7mBbYa3WafIlb0MJLEvRFJOPHLmuBRUcISWt0pHzGmBxNT_/w400-h384/InspectObjectInCustomizeDialog.png" width="400" /></a></td></tr><tr><td class="tr-caption" style="text-align: center;">Figure 3: "Inspect Object" command in "Customize" dialog </td></tr></tbody></table><p><br />In addition to showing the selected object, I have added a UNO command named “Inspect Object” (.uno:InspectSelectedObject), which can be added to context menus for objects (See Figure 3). The purpose of this command is to provide a way to open the DevTools docking window and automatically show the current selected object. If a user regularly uses the object inspector, this may be a more convenient way for them to access DevTools. Note that by default the command isn't added to any context menu, this is up to the user. However, if there will be demand to add this to context menus, it can be easily added. </p><table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto;"><tbody><tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhzw_H2ZgJ4RDm-DF5_aZe6-YOQB-ADI2N-_91IPta4lQ7qwISogf6zjlQUn3tQzs3vROrl-I3u_bV5rrG3UtMcFCW4Q5fWnMVQdsXzkM8ULeNosvq4MPcqXe5AfK5blRXUK98-8ALrNpuT/s1157/InspectObjectContextMenu.png" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="1157" data-original-width="824" height="400" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhzw_H2ZgJ4RDm-DF5_aZe6-YOQB-ADI2N-_91IPta4lQ7qwISogf6zjlQUn3tQzs3vROrl-I3u_bV5rrG3UtMcFCW4Q5fWnMVQdsXzkM8ULeNosvq4MPcqXe5AfK5blRXUK98-8ALrNpuT/w285-h400/InspectObjectContextMenu.png" width="285" /></a></td></tr><tr><td class="tr-caption" style="text-align: center;">Figure 4: "Inspect Object" context menu entry on a shape object</td></tr></tbody></table><p>The example in Figure 4 shows the context menu of a shape object, where the selected entry is the added "Inspect Object". </p><p>From the implementation standpoint, it was necessary to move the whole DevTools from svx to sfx2 module. This was mainly necessary to get .uno:InspectSelectedObject to work, because we need to signal to DevTools that we want to show the current selection and not the document root in the object inspector. Because the svx depends on sfx2 module, it is not possible to access svx from sfx2 (only the other way around). </p><h1 style="text-align: left;">Improvements to object inspector</h1><div>The object inspector was previously a single tree view only, which had services, interfaces, properties and methods categories as root tree entries. This has now been changed so that the categories are now pages in a tab view, and each category has its own tree view (can be seen in Figure 2). The main problem with one tree view is that columns for each of the categories are different. For example, the properties category has object, value and type categories but the same columns make no sense for methods (which has return type and input parameters). </div><div><br /></div><div>For methods it now shows the method name, return type and parameters. The types are currently simplified types, which are easier to read (instead of exact type name of the object it just writes "object"), but the user will want to know the exact type too, so this is a WIP.</div><div><br /></div><div>For properties it shows the type and value of the property, and it is possible to expand a property if the type is a complex type (object, struct) so it lists nested properties. If the value is an enum, then we get the name of the enum value automatically and show the name instead. </div><div><br /></div><div>Support for sequences was also added, so the sequence can be expanded and a list of indices and values is presented. If the current object supports XNameAccess or XIndexAccess, the names and indices are added into the property list, so the user can navigate to those. </div><div><br /></div><div>With this additions, it is already easier to inspect objects than it previously was using the Xray tool, and I'm sure it will get even better when it is finished. </div><h1 style="text-align: left;">Next steps</h1><div>The object inspector is already in a very good shape so I encourage everyone to try it and give feedback, what can be improved, changed or added - especially if you use Xray or MRI regularly. </div><div><br /></div><div>For the next steps the major focus will be to fix a couple of bugs and crashes (mainly due to missing checks if objects are available), work on the UI, object stack (so it is possible to go back to the previous object) and finalizing all the features of the object inspector. </div><h1 style="text-align: left;">Credits</h1><div>Many thanks to <a href="https://www.documentfoundation.org/">TDF</a> and users that support the foundation by providing donations, to make this work possible. </div><div><br /></div><div><b>To be continued...</b></div>Tomaž Vajngerlhttp://www.blogger.com/profile/17183461757847157603noreply@blogger.com2tag:blogger.com,1999:blog-2640003666891728724.post-16849727683096992492021-01-21T14:39:00.000+01:002021-01-21T14:39:50.623+01:00Built-in "Xray" like UNO object inspector – Part 1<p>
When developing macros and extensions in LibreOffice it is very useful to have an object inspector. With that tool you inspect the object tree and the methods, properties and interfaces of
individual objects to understand its structure. There are quite some different object inspectors available for LibreOffice as extension. Probably the best known is called “XrayTool”, but
there were also others like MRI, which was build for a similar purpose and
various other more simple object inspectors (for example one is provided as an
code example in ODK).
</p>
<p>
As a tool like this is very valuable it makes sense that it would be provided
out-of-the-box by LibreOffice without the need for the user to get it
separately. Also the user could even be unaware the such a tool exists. For
this reasons
<a href="https://www.documentfoundation.org/">The Document Foundation</a>
(TDF) put up a tender to
<a href="https://blog.documentfoundation.org/blog/2020/07/27/tender-for-implementing-support-for-a-dedicated-built-in-uno-object-inspection-tool-in-libreoffice-202007-02/">create a built-in Xray like UNO object inspector</a>, which was awarded to
<a href="https://www.collaboraoffice.com/">Collabora</a> and we are now in the
process of implementing it.
</p>
<p>
Thank you TDF and the donating users of LibreOffice to make the work on this
tool possible.
</p>
<h2 style="text-align: left;">The Plan</h2>
<p>
The major gripe with Xray and other tools is that the user needs to go into
the macro editor and run a the script with the inspecting object as the
parameter.
</p>
<p>For example to inspect the root UNO object:</p>
<p><span style="font-family: courier;">Xray ThisComponent</span></p>
<p>or for example to inspect a specific (first) Calc sheet:</p>
<p>
<span style="font-family: courier;">Document = ThisComponent<br /></span><span style="font-family: courier;">AllSheets = Document.getSheets()<br /></span><span style="font-family: courier;">MySheet = AllSheets.getByIndex(0)<br /></span><span style="font-family: courier;">Xray MySheet</span>
</p>
<p></p>
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto;">
<tbody>
<tr>
<td style="text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg1mfwJJ5v50aBWBgfRBadgbAYQ68kcFhyBKBOYfZBYGm41IHmKJftBvbSWX03-fXZtbCeWnwurnxfHh_NmToG0ju522TvNDEDh5njyUIIybQjaCuFsbqAdizkd43pIyr5u5jNbiWfSEk5N/s2210/WatchWindow.png" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="2210" data-original-width="2024" height="640" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg1mfwJJ5v50aBWBgfRBadgbAYQ68kcFhyBKBOYfZBYGm41IHmKJftBvbSWX03-fXZtbCeWnwurnxfHh_NmToG0ju522TvNDEDh5njyUIIybQjaCuFsbqAdizkd43pIyr5u5jNbiWfSEk5N/w586-h640/WatchWindow.png" width="586" /></a>
</td>
</tr>
<tr>
<td class="tr-caption" style="text-align: center;">
Figure 1: Watch window in the macro editor
</td>
</tr>
</tbody>
</table>
<p>
We can do much better than this. So the idea is to have a object inspector
tool as a dockable bottom widget, very similar to “developer tools” that you
can find in popular web browsers.
</p>
<p>
The left hand side of the tool will have a subset of the document model (DOM)
presented as a tree and on the right hand side a object inspector view that
will show the current UNO object’s properties, methods in a tree view that is
taken from the Watch window in the LibreOffice macro editor (see Figure 1).
When the user would select an UNO object in the DOM tree view (left-hand
side), the object inspector (right-hand side) would show it.
</p>
<p>
An additional functionality that is present in developer tools in web browsers
is the ability to point & click an object directly in the document, which
can then be inspected in the developer tool. A similar functionality to this
will be possible in LibreOffice, where you will be able to select an object in
the document, that is then be shown in the object inspector (right-hand side).
</p>
<h2 style="text-align: left;">Implementation so-far</h2>
<p>
The implementation has been divided into 4 parts:
</p>
<p></p>
<ol style="text-align: left;">
<li>
Introduce a new dockable window on the bottom of the UI.
</li>
<li>DOM tree view (left-hand side)</li>
<li>Implement point&click functionality.</li>
<li>Object inspector view (right-hand side)</li>
</ol>
<p></p>
<p>
Currently the 1. and 2. part have been implemented, so this blog post serves
as a report of the work that has been done until now. For 3. and 4. part, a
limited subset of the functionality has been already implemented, but it
is not yet expected to work correctly and may change a lot in the future.
</p>
<h3 style="text-align: left;">
Dockable window
</h3>
<div>
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto;">
<tbody>
<tr>
<td style="text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi011zBQ34ShF4HIv6fhJ2aHMwvhY97QtQ5bhPlkfNkCj5Sjk7G1YLHvL1L8g8t8Rk4-3EIzwX3hYxCrdjzhiqLAgND3Eq51Ms-E2iKUWVo_wd6407QSB8lWQfDOPD47UtXNxjEmx1OgByl/s774/DevelopmentToolsInMenu.png" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="774" data-original-width="744" height="400" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi011zBQ34ShF4HIv6fhJ2aHMwvhY97QtQ5bhPlkfNkCj5Sjk7G1YLHvL1L8g8t8Rk4-3EIzwX3hYxCrdjzhiqLAgND3Eq51Ms-E2iKUWVo_wd6407QSB8lWQfDOPD47UtXNxjEmx1OgByl/w385-h400/DevelopmentToolsInMenu.png" width="385" /></a>
</td>
</tr>
<tr>
<td class="tr-caption" style="text-align: center;">
Figure 2: Development Tool in the menu
</td>
</tr>
</tbody>
</table>
</div>
<p>
The dockable window has been added to all components of LibreOffice, which can
be enabled in the menu (see Figure 2) under Help / Development tool (this
location and the name can still change). The development tool code is all contained under
<span style="font-family: courier;">svx/source/devtools/</span>
<span style="font-family: inherit;">
where the docking window is implemented in </span><span style="font-family: courier;">DevelopmentToolDockingWindow.cxx</span><span style="font-family: inherit;">.</span></p>
<h3 style="text-align: left;">
<span style="font-family: inherit;">DOM tree view (Left-hand)</span>
</h3>
<div>
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto;">
<tbody>
<tr>
<td style="text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEizHVB1tmyz8PGKIO_jaeiHgAWcjfrf7-PFg-aK174yloLsgrwtwCYSgIdx_QaiM41r3geGCIGeJD2puOlENGibIXSVbn_J5piQtfbN_ILf76FXOPd9vrEAcLInNMRYL730wSGOApTwz5aa/s2010/DevelopmentTool.png" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="2000" data-original-width="2010" height="637" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEizHVB1tmyz8PGKIO_jaeiHgAWcjfrf7-PFg-aK174yloLsgrwtwCYSgIdx_QaiM41r3geGCIGeJD2puOlENGibIXSVbn_J5piQtfbN_ILf76FXOPd9vrEAcLInNMRYL730wSGOApTwz5aa/w640-h637/DevelopmentTool.png" width="640" /></a>
</td>
</tr>
<tr>
<td class="tr-caption" style="text-align: center;">
Figure 3: Development tool dockable window
</td>
</tr>
</tbody>
</table>
<span style="font-family: inherit;"><br /></span>
</div>
<p>
The idea of the DOM tree view is to provide a useful subset of the DOM
objects, that the user can quickly access and inspect. Without this, the
users would have to traverse to the objects they are interested in inside
the object inspector itself, which may not be as straight forward to the
users that aren’t familiar with the DOM, and even if they are, the DOM
subset still makes it easier.
</p>
<p>
The DOM tree view as currently implemented (see Figure 3), has a root
“Document” element always available, which represents the root document UNO
object, from where you can traverse to any other relevant UNO object in the
DOM tree. Other available objects depend on the component.
</p>
<p>
In Writer the available objects are:
</p>
<ul style="text-align: left;">
<li>Paragraphs</li>
<li>Shapes</li>
<li>Tables</li>
<li>Frames</li>
<li>Graphic Objects</li>
<li>Embedded Objects (OLE)</li>
<li>Style Families & Styles</li>
</ul>
<p></p>
<p>In Calc:</p>
<ul style="text-align: left;">
<li>Sheets</li>
<li>Shapes (per sheet)</li>
<li>Charts (per sheet)</li>
<li>Pivot Tables (per sheet)</li>
<li>Style Families & Styles</li>
</ul>
<p></p>
<p>In Impress and Draw:</p>
<ul style="text-align: left;">
<li>Pages / Slides</li>
<li>Shapes (per page / slide)</li>
<li>Master Slides</li>
<li>Style Families & Styles</li>
</ul>
<p></p>
<p>
If there are other object(s) you would like to see in the DOM tree view, please let me know.</p>
<p>
The main implementation file for the left-side DOM tree view is <span style="font-family: courier;">DocumentModelTreeHandler.cxx</span><span style="font-family: inherit;"> in the </span><span style="font-family: courier;">svx/source/devtools/</span><span style="font-family: inherit;">
folder. For each node of the tree view, there is an object attached
(subclass </span><span style="font-family: courier;">DocumentModelTreeEntry</span><span style="font-family: inherit;">) to the node, which is responsible to fill the child nodes and provide the
UNO object of the current node. The whole DOM tree view is populated
on-demand, when the tree is expanded. This makes sure that we don’t take a
lot of time inserting the whole object tree before-hand and have a more
up-to-date view of the UNO objects. </span>
</p>
<p>
All the code is present in the LibreOffice master repository. Please try out
the development tools in a recent daily build. Any comments or suggestion are
welcome.
</p>
<h2 style="text-align: left;">
Next part - point&click functionality
</h2>
<p>
Partial support for the point&click has already been added. There is a
selection listener added, which remembers what the selected object in the
document is. This object is the available in the DOM tree view under the name
“Current Selection”. If the user clicks on the node in the tree view, then the
right-hand object inspector shows the UNO object.
</p>
<p></p>
<p></p>
<p>To be continued..</p>
Tomaž Vajngerlhttp://www.blogger.com/profile/17183461757847157603noreply@blogger.com5tag:blogger.com,1999:blog-2640003666891728724.post-25600862275637392972020-10-13T23:36:00.000+02:002020-10-13T23:36:53.381+02:00PDF annotations support<p>In Draw, a PDF document can be opened using the PDFium library for rendering, where each page in the Draw document contains an rendered image from the PDF. This mode is useful for viewing PDFs and allows for the best fidelity. With viewing, there is also a need to review and comment and this is where PDF annotations come in as adding the support for the PDF annotations and to support a review based workflow has been one of my recent task at Collabora Productivity.</p><p>PDF supports a wide variety of annotations, but we don't support all of them in Draw. What we do support are comments, which are similar to pop-up note annotations in PDF, so the easiest is to add those first. To be able to use pop-up notes in Draw, we need to import them. This is done at import by using PDFium after we created the PDF graphic for rendering. In Draw, we then insert this as comments and so we get the basic support for manipulating with annotations, but how to save the changes? PDF export already supports saving comments as annotations, so this mostly already works (I needed to fix some bugs and add support for saving all needed properties).</p><p><br /><table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto;"><tbody><tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhKQ4F4Yo1NlIMql2c-IZcGRF_Fjfz8f6SXiLb-J9RoWpIJNDoIC9qQRyK_fRge0xys6s01pPxQ8TiyAExapr6q6__vJppCFEMih1hCQT3CQx_rJ5WdtsQUus7MZsMX6y2eebQ3ft4X2EnZ/s1920/PDFAnnotations1.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="1200" data-original-width="1920" height="250" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhKQ4F4Yo1NlIMql2c-IZcGRF_Fjfz8f6SXiLb-J9RoWpIJNDoIC9qQRyK_fRge0xys6s01pPxQ8TiyAExapr6q6__vJppCFEMih1hCQT3CQx_rJ5WdtsQUus7MZsMX6y2eebQ3ft4X2EnZ/w400-h250/PDFAnnotations1.png" width="400" /></a></td></tr><tr><td class="tr-caption" style="text-align: center;">Figure 1. Pop-up Note annotation in PDF viewer (Evince) and Draw</td></tr></tbody></table><br /></p><p>How the output compares between a PDF viewer and inside LibreOffice can be seen in Figure 1. All this is available in LibreOffice master and should be included in LibreOffice 7.1.</p><p>What about other annotations that are supported in PDF? The work to add those is ongoing and the recent success has been adding support to draw vector graphic annotations like polygon, ink (freehand), squares (rectangles) and circles (ellipses). So the idea is that instead of the usual marker for a comment, we draw a vector graphic that we read from the PDF annotation. </p><p>The first thing I needed to do is to extend the PDFium library, which didn't support reading all polygon vertices and ink strokes from the document. What also wasn't supported is reading the border information, which is needed for line widths.</p><p>Next thing is extending the import to read the geometry data, so I dded a special PDFAnnotationMarker class for that. Then the geometry data needs to be stored on the sd::Annotation class (implementation of XAnnotation, but I didn't extend XAnnotation at this point as it wasn't needed - yet) . Drawing is performed in AnnotationTag, where we create a new OverlayPolyPolygon object, that is responsible for creating the Primitive2D for the marker. What we get after this is done is shown in figure 2.</p><p> <table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto;"><tbody><tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg1G1tNZBuRkgZvoeP_CZd9kpOQOD8bMTKEcei0PXBqqOG75BhIb0CsXcd0S4CK7LHzF7OgnBZ9RI0-ekTiHpukSRXr5eDUezzun_Bk-nJj-Ejy6ggi8DGWQlTADGTu8IGdiWyHm_JdDDUB/s1920/PDFAnnotations2.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="1200" data-original-width="1920" height="250" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg1G1tNZBuRkgZvoeP_CZd9kpOQOD8bMTKEcei0PXBqqOG75BhIb0CsXcd0S4CK7LHzF7OgnBZ9RI0-ekTiHpukSRXr5eDUezzun_Bk-nJj-Ejy6ggi8DGWQlTADGTu8IGdiWyHm_JdDDUB/w400-h250/PDFAnnotations2.png" width="400" /></a></td></tr><tr><td class="tr-caption" style="text-align: center;">Figure 2. Multiple annotations in PDF Viewer and Draw<br /></td></tr></tbody></table><br /></p><p>This work will shortly be merged to the LibreOffice master. </p>Tomaž Vajngerlhttp://www.blogger.com/profile/17183461757847157603noreply@blogger.com02000 Maribor, Slovenia46.5546503 15.645881217.013341793115181 -19.510368800000002 76.095958806884823 50.8021312tag:blogger.com,1999:blog-2640003666891728724.post-69555844244489658002020-01-22T09:54:00.000+01:002020-01-22T09:54:11.567+01:00Accessibility checker and support for PDF/UA specs <a href="https://en.wikipedia.org/wiki/PDF/UA">PDF/UA</a> or ISO 14289 is a specifications that defines the requirements for accessibility in a PDF document. The specification defines the required structure and formatting of the document (also refers to <a href="https://www.w3.org/TR/WCAG21/">WCAG</a> specification from W3C for use on the web) and PDF features, which should be enabled or disabled so the document is better suited for accessibility (for example PDF tags are required).<br />
<br />
Thanks to the Dutch Standardisation Forum for financially sponsoring and <a href="https://www.collaboraoffice.com/">Collabora Productivity</a> in cooperation with <a href="https://www.nouenoff.nl/">Nou&Off</a> for the work on implementing this specification into LibreOffice.<br />
<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh_3vL82tcuWzH_PIizDt9W-gPxCMvvzQQxcmcakZ7H7hutfSYt_uIQ3l_YEdglksCs9Sckv98vKeo9f11BcDbFhiAZaPlx1fXgqxWaTI1pZzaLJqXKLCZ-IMxRXOV_ao1onJoxWGWWWg9n/s1600/PDF+Export+Dialog.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="1189" data-original-width="1600" height="296" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh_3vL82tcuWzH_PIizDt9W-gPxCMvvzQQxcmcakZ7H7hutfSYt_uIQ3l_YEdglksCs9Sckv98vKeo9f11BcDbFhiAZaPlx1fXgqxWaTI1pZzaLJqXKLCZ-IMxRXOV_ao1onJoxWGWWWg9n/s400/PDF+Export+Dialog.png" style="cursor: move;" width="400" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">Figure 1: PDF Export</td></tr>
</tbody></table>
The implementation in LibreOffice is currently done only for Writer. It consists of two parts.: First is to enable PDF/UA support into PDF export, which writes a PDF/UA flag into the PDF document and enables all the required features. The PDF export dialog was extended with a checkbox (see figure 1).<br />
<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhHxitQGO3rsL7ROg8T1QRWmOloB78qMXM5WBUeCVV2hqgyHyf_OfMgMf5mo7MvYX689C08xQ1HsaeIpAABgzjiRRnpcXbbfJCrUclxW-mfQMBH1Cj0FHZA0CNiBHlPLCxk3jJIhNDuuGUs/s1600/Menu+-+Acessibility+Check.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="624" data-original-width="1181" height="211" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhHxitQGO3rsL7ROg8T1QRWmOloB78qMXM5WBUeCVV2hqgyHyf_OfMgMf5mo7MvYX689C08xQ1HsaeIpAABgzjiRRnpcXbbfJCrUclxW-mfQMBH1Cj0FHZA0CNiBHlPLCxk3jJIhNDuuGUs/s400/Menu+-+Acessibility+Check.png" width="400" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">Figure 2: Accessibility Check in menu </td></tr>
</tbody></table>
<br />
The second part is an accessibility check functionality, which traverses the document structure and gather all possible accessibility issues. The accessibility check can be run manually through the menu under Tools - Accessibility Checker (see figure 2), or it will be triggered after PDF export dialog, if the PDF/UA support is enabled. The accessibility issues are presented in a dialog (see figure 3).<br />
<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiPFkFruEUIwGwkkEPCRjWBhYflbGjHCJjlhBZJWrGz7XQzvo5FE9VIbRT8Kzm_m7ZshDGHYo8fwyRL2bmlM8QXdggjBYOLqwyI-ozolYewTC_YmDgB2Sd69ivSdxc3LPVNaq__bpaLQFpN/s1600/Accessibility+Check+Dialog.jpeg" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="1113" data-original-width="1600" height="277" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiPFkFruEUIwGwkkEPCRjWBhYflbGjHCJjlhBZJWrGz7XQzvo5FE9VIbRT8Kzm_m7ZshDGHYo8fwyRL2bmlM8QXdggjBYOLqwyI-ozolYewTC_YmDgB2Sd69ivSdxc3LPVNaq__bpaLQFpN/s400/Accessibility+Check+Dialog.jpeg" width="400" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">Figure 3: Accessibility check dialog</td></tr>
</tbody></table>
<br />
The checks that are (currently) implemented are:<br />
<br />
<ul>
<li>Check that the document title is set.</li>
<li>Check that the document language is set, or that all styles that are in use, have the language set.</li>
<li>Check all images, graphics, OLE objects for the alt (or title in some objects) text.</li>
<li>Check tables don't include splits or merges, which aren't allowed by the specifications. The table should be </li>
<li>Check for fake/manual numbering (not using integrated numbering). For example writing "1." "2." "3." at the beginning of the paragraphs. </li>
<li>Check that hyperlink text is not a hyperlink itself - hyperlink should be described. </li>
<li>Check for the contrast between text and the background. The algorithm is described in the WCAG specification. </li>
<li>Check for blinking text, discouraged for the obvious reasons.</li>
<li>Check for footnotes and endnotes, which should be avoided.</li>
<li>Check for heading order. Order of the headings must increase incrementally with no skips (for example Heading 1 to Heading 3, skipping Heading 2).</li>
<li>Check, if text conveys additional meaning with (direct) formatting.</li>
</ul>
<br />
The PDF/UA support will be included in LibreOffice 7.Tomaž Vajngerlhttp://www.blogger.com/profile/17183461757847157603noreply@blogger.com5tag:blogger.com,1999:blog-2640003666891728724.post-29530739907732758292018-04-23T16:59:00.002+02:002018-04-23T16:59:26.056+02:00Improving the image handling in LibreOffice - Part 3<h2>
GraphicObject refactoring</h2>
<div>
<br /></div>
GraphicObject and the implementation of XGraphicObject (UnoGraphicObject) and XGraphic (UnoGraphic) were located in module svtools, which is hierarchically above vcl. This is problematic when creating new instances like in Graphic.GetXGraphic method, which needs to bend backward to make it even work (ugly hack by sending the pointer value as URL string to GraphicProvider). The solution to this is to move all GraphicObject related things to vcl, which surprisingly didn't cause a lot problem and once done, it looks like a much more "natural" place.<br />
<br />
Regarding the UNO API of XGraphicObject - what is left to do here is to properly clean up the uniqueID, as it is not possible to use it anymore for anything else as a uniqueID (used only in filters for the image names, if the name is not yet known).<br />
<br />
<h2>
Managing memory used by images</h2>
<div>
<br /></div>
<div>
<h2>
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgDfoprmpTKUYDnMJMAHnaXbtwp7VVJW0MAqWHMe82rL9AA4HPb5u-jWSB_JwkdSqt4j_Y8sarIIBrN2xAO_GZBP61liH1biMppUcFuYFx_feegpYTS-cCpcHFZGBE_LdYEWnK6G1z9569g/s1600/GraphicObjectHierarchy_Old.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" height="277" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgDfoprmpTKUYDnMJMAHnaXbtwp7VVJW0MAqWHMe82rL9AA4HPb5u-jWSB_JwkdSqt4j_Y8sarIIBrN2xAO_GZBP61liH1biMppUcFuYFx_feegpYTS-cCpcHFZGBE_LdYEWnK6G1z9569g/s400/GraphicObjectHierarchy_Old.png" width="400" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;"><table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td class="tr-caption" style="font-size: 12.8px;">Figure1: Hierarchy before refactoring</td></tr>
</tbody></table>
<div style="font-size: medium; text-align: start;">
</div>
</td></tr>
</tbody></table>
</h2>
<br />
Previously the memory managing was done on the level of GraphicObjects, where a GraphicManager and GraphicCache (see figure 1) were responsible to create new instances from uniqueID and manage the memory usage that GraphicObject take. This is not possible anymore as we don't operate with uniqueIDs anymore, but always use Graphic and XGraphic objects (in UNO), so we need to manage the creation of Graphic object or more precisely - ImpGraphic (Graphic objects are just ref. counted objects of ImpGraphic). </div>
<div>
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj-N6upwGlw_rCKRqd8EqDbnKX8YEKsOH7NiQtr4bRUSkXQMJPPHt7QVZe9ZLijxPLICKP9aXWyjS2iOvSSsF88LBSAsBnyEVuKbujII5zF1_5CwbujoB0nDp7m9T6eDB2elrWhIooQDL3-/s1600/GraphicObjectHierarchy_New.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" height="400" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj-N6upwGlw_rCKRqd8EqDbnKX8YEKsOH7NiQtr4bRUSkXQMJPPHt7QVZe9ZLijxPLICKP9aXWyjS2iOvSSsF88LBSAsBnyEVuKbujII5zF1_5CwbujoB0nDp7m9T6eDB2elrWhIooQDL3-/s400/GraphicObjectHierarchy_New.png" width="345" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;"><table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td class="tr-caption" style="font-size: 12.8px;">Figure 2: Hierarchy after refactoring</td></tr>
</tbody></table>
</td></tr>
</tbody></table>
So to make this possible GraphicManager and GraphicCache need to be decoupled and removed from GraphicObject and a new manager needs to be introduced between Graphic and ImpGraphic, where the manager controls the creation and accounts for the memory usage (see Figure 2).</div>
<div>
<br />
<h2>
Graphic swapping and swapping strategy</h2>
</div>
<div>
<br /></div>
<div>
In the To release the memory of graphic objects, we swap them out to a temp file and read back (swap-in) when we need them again. In the previous implementation this was partially directed by the SdrGrafObj (common image implementation) and SwGrfNode (Writer image implementation). For each graphic object there was a timer when to trigger an automatic swap-out + the swap-out that can happen when a memory limit is exceeded.<br />
<br />
For the new code external swapping directed from SdrGrafObj and SwGrfNode was removed, so they can't influence when swapping will happen (maybe in the future they can provide hints when it is a good time to do swapping). There is now a global timer which triggers checking of all Graphic objects if any of them can be swapped out in case we exceed memory limit. Same code is triggered when a new object is created too. A object will be swapped out if it is not used for a certain amount of time. Each object tracks the timestamp when it was last used.<br />
<br />
A swap-in happens if the object is swapped-out (obviously) and certain data is needed (under-laying bitmap, animation or metafile). This is checked at the same code-path where the timestamp updating happens.<br />
<br />
The new swapping strategy is relatively simple - if a lot of memory is needed by graphic objects in a certain time, we let it use it and don't try to over-aggressively try to free it. In the past this cased swap-out and swap-in cycle that made the application completely unusable. In the future, external hints when a certain Graphic object can be swapped out may be added, so we can perform swapping more effectively. There are also several other ideas which will increase performance and reduce memory usage that can be implemented now with the new hierarchy where most all of the swapping is contained inside the Graphic itself, but all of this is currently out of the scope of this work.<br />
<br /></div>
<h2>
Other changes to Graphic</h2>
<div>
<br /></div>
<div>
Another changes to Graphic done were related to lazy loading. When a document is loaded, we don't want to load Graphic into memory, if it is not needed yet (for example we display the first page but the graphic is on page 10). In document filters (ODF for example) we previously transported the URL of an external or internal graphic to the document model, where it was lazily loaded when it was actually needed. This is not possible now anymore as we need to create a XGraphic object already in the document filter. To overcome this we need to to have an unloaded Graphic, which is created already in a swapped-out state and swapped-in when needed.<br />
<br />
The GraphicFilter didn't allow something like this, so I needed to add a new method, which doesn't actually load the image, but just gathers what kind of the image is loaded and its metadata (image size) and creates a GfxLink object that includes the (compressed) image data. The metadata is needed as we don't want to actually force a load when this basic information is requested. Actually we want to load the image as late as this is possible.<br />
<br />
Another issue is also that we can have an external image (loaded from a file or even URL on the internet). The issue is similar to the lazy loading scenario, but it is different that a Graphic now must know the URL with which it was created and can be created completely empty (no loading of any kind). The reason for this is that loading is directed by the LinkManager, which is part of the document model. For security reasons the LinkManager can not allow that a Graphic is loaded so loading is directed by the LinkManager on demand (first usage). LinkManager also takes care of all URLs of various external resources. The user can look at those resources and change the URL of them or trigger an update. Changing URL and updating an object was previously done in SdrGrafObj and SwGrfNode, but now this is moved to the common code in Graphic object where SdrGrafObj and SwGrfNode only direct what to do. There are still rooms to improve things here, however not the scope of this work.<br />
<br /></div>
<div>
<h2>
Next steps</h2>
<div>
<br /></div>
<div>
Finishing up this work by revising the UNO API and fixing known bugs.</div>
<div>
<br /></div>
<h2>
Credits</h2>
<div>
<br /></div>
Many thanks to <a href="https://www.collaboraoffice.com/">Collabora Productivity</a>, <a href="https://www.documentfoundation.org/">TDF</a> and users that support the foundation by providing donations, to make this work possible.<br />
<br />
<b><i>To be continued...</i></b></div>
Tomaž Vajngerlhttp://www.blogger.com/profile/17183461757847157603noreply@blogger.com0tag:blogger.com,1999:blog-2640003666891728724.post-72038745101497715522018-03-13T10:30:00.001+01:002018-03-13T10:30:37.730+01:00Improving the image handling in LibreOffice - Part 2It's been some times from my last blog post and in that time I continued with refactoring the code to get rid of use of GraphicObject uniqueID being passed around and stored in the model. The state of the code now is looking fine as we almost don't use the uniqueID anymore, which means that I can start with the next step of Graphic and GraphicObject improvements. <br />
<br />
There is a thing I forgot to clarify in the last blog post and this is that the GraphicObject uniqueID is usually passed around in the form of a URL string. The string has the prefix "vnd.sun.star.GraphicObject" and followed by the GraphicObject uniqueID. Using that URL it is possible to re-create a GraphicObject by passing the unique ID as the construction parameter (see constructor with OUString parameter on GraphicObject or UNO serviceGraphicObject::createWithId).<br />
<br />
<h2>
Usage in filters</h2>
The most "heavy" users of the uniqueID were the document format filters (xmloff, oox & writerfilter) which generally use it to read the images from the storage (usually ZIP) and convert the GraphicObject and pass GraphicObject uniqueID around. At writing it does the reverse, get the GraphicObject URL and "resolve" the URL to the package URL. At conversion the GraphicObject is created and the image is stored into the storage. To do this there XGraphicObjectResolver published UNO interface which has only resolveGraphicObjectURL which converts a GraphicObject URL to the Package URL and back. <br />
<br />
Resolving the URL is not the correct approach anymore so I had to do it in a different way. The result of that is XGraphicStorageHandler, which has explicit method to load and save an XGraphic from the package URL, which does everything without the need to use the GraphicObject unique ID.<br />
<br />
In addition a graphic can also be external - somewhere on the disk or internet, identified by an external URL. For this case I implemented a GraphicLoader, which is generally just uses XGraphicProvider to load the graphic (in one of the next steps this will be reversed so that XGraphicProvider is just a UNO interface that uses GraphicLoader).<br />
<br />
The special case with external URLs is also that we need to remember the URL, which was used to load the graphic, so that we can later just save the URL and not the Graphic into the storage. Previously the URL was always passed along as string so this wasn't a problem, but now we pass XGraphic. So for this I had to extend the Graphic in VCL with an origin URL attribute, to solve this use case. In a next steps the URL loading will be extended even more so the Graphic itself will handle URL completely transparently to the outside.<br />
<br />
<h2>
UNO properties</h2>
<div>
Usually the filters used the UNO API to set the GraphicObject unique ID into the document model. This was mostly implemented as a properties on various interfaces in UNO. Mostly used name of the properties was GraphicURL (used in different places), but there were also other properties: </div>
<div>
<ul>
<li>BackGraphicURL (for backgrounds)</li>
<li>HeaderBackGraphicURL (for backgrounds in header)</li>
<li>FooterBackGraphicURL (for backgrounds in footer)</li>
<li>ParaBackGraphicURL (for backgrounds in paragraphs)</li>
<li>ThumbnailURL (for thumbnail of a graphic - not in IDL)</li>
<li>ReplacementGraphicURL (for replacement graphic - not in IDL)</li>
<li>FillBitmapURL (BitmapTable)</li>
</ul>
</div>
<div>
All these properties are now deprecated and removed and an alternative was added (where needed) that uses the XGraphic or XBitmap types (they use the same implementation so either can be used). This was done as following:</div>
<div>
<div>
<ul>
<li>GraphicURL -> Graphic (type XGraphic) and GraphicBitmap (type XBitmap) for bullets</li>
<li>BackGraphicURL -> BackGraphic (type XGraphic) </li>
<li>HeaderBackGraphicURL -> HeaderBackGraphic (type XGraphic)</li>
<li>FooterBackGraphicURL -> FooterBackGraphic (type XGraphic)</li>
<li>ParaBackGraphicURL -> ParaBackGraphic (type XGraphic)</li>
<li>ThumbnailURL -> Thumbnail (type XGraphic)</li>
<li>ReplacementGraphicURL -> ReplacementGraphic (type XGraphic)</li>
<li>FillBitmapURL -> FillBitmap (type XBitmap)</li>
</ul>
</div>
</div>
<div>
<div>
There is also ImageURL which is used in form controls, but this was still left inside as there is already a Graphic property which is an alternative.</div>
</div>
<div>
<br /></div>
<div>
As the GraphicObject URL is going away (they won't be created anymore and won't be possible to get the GraphicObject back using the URL), so will the properties, as the content of them won't make much sense anymore.</div>
<div>
<br /></div>
<h2>
Next steps</h2>
<div>
The next step is now to finally work on Graphic itself, which I'm much more excited about. The managing of memory (GraphicManager) will move from the GraphicObject to the Graphic itself. When a new Graphic is created, the original bit-stream needs to be saved immediately to the temp folder, where the Graphic can always load the image again and is always free to release the memory if needed (this also means it won't need to load the image to the memory until it is actually needed). With this I think that handling of images will finally be a lot more predictable and homogeneous (no different implementations of things through different modules) and we can actually introduce new features in the future much more easily. </div>
<div>
<br /></div>
<div>
Back to work...</div>
<h2>
Credits</h2>
<div>
Many thanks to Collabora Productivity, TDF and users that support the foundation by providing donations, to make this work possible.</div>
Tomaž Vajngerlhttp://www.blogger.com/profile/17183461757847157603noreply@blogger.com0tag:blogger.com,1999:blog-2640003666891728724.post-69842586954199170942018-01-31T12:10:00.002+01:002018-01-31T12:10:16.418+01:00Improving the image handling in LibreOffice - Part 1<h2>
Prologue</h2>
It is known for some time that the image life-cycle in LibreOffice is problematic and can potentially lead to image loss, but to make the life-cycle more robust against loss, a lot of refactoring would need to be done. Another issue is also the mechanism of images swapping in and out of the memory. Keeping images in memory takes a lot of space so when a certain amount is hit, the images get swapped to disk and memory is freed. The problem is that it can happen that the cache handler starts constantly to swap images in and out (especially with with multi-megapixel images that are the norm today) and LibreOffice stalls to halt.<br />
<br />
Because of this issues, TDF put up a <a href="https://blog.documentfoundation.org/blog/2017/05/02/tender-improve-image-handling-libreoffice-201705-01/">tender</a> to improve the situation with image handling and Collabora Productivity was <a href="https://blog.documentfoundation.org/blog/2018/01/29/tdf-uses-tendering-process-improve-libreoffice-share-knowledge-community/">selected</a> to implement it, and I will do the development work.<br />
<h2>
<br /></h2>
<h2>
Problems with the image life-cycle - detailed</h2>
<div>
Currently, when an image is read from a document, a GraphicObject is created for the image and handled over to the GraphicManager which manages the life-cycle. When this happens we usually get back the string based unique ID of the GraphicObject with which we can always get access the image by creating a new GraphicObject with the unique ID (GraphicManager will look for the image with that unique ID). Usually the unique ID is the one that is passed on between layers in LibreOffice (for example from ODF filter when loaded, to the model, where it is manipulated and then to the OOXML filter when saving) but the unique ID itself is just a "reference" to the image and by itself it doesn't have any control over when the image can safely be removed and when not. It could happen that in a certain situation we would still have the unique ID referenced somewhere in the model, but the image would already be removed. This is dangerous and needs to be changed. </div>
<div>
Usually for this kind of object we use reference counting technique, where we pass a objects around that holds a reference to the object resource. When the object is created, the reference count is increased, when destroyed, the reference count is decreased, when the reference count reaches zero, the resource object is destroyed.</div>
<div>
<h2>
<br /></h2>
<h2>
The solution for the life-cycle</h2>
</div>
<div>
So instead of passing around of unique ID the idea is to use the usual reference counting technique, that is normally used in this situation. The GraphicObject in mainly a wrapper around Graphic (which then holds a pixel-based image, or animated image, or possibly a vector image), and in addition it keeps additional attributes (gamma, crop, transparency, ...). It also has the implementation of swapping-in and out (but I'll explain this another time). On the other hand Graphic is properly reference-counted already (Graphic objects are reference counting the private ImpGraphic) so the solution to the life-cycle problem is that instead of GraphicObject unique ID we would just pass along the Graphic object instead, or XGraphic, XBitmap which are just UNO wrappers around Graphic. Potentially we could also pass along the GraphicObject or XGraphicObject (UNO wrapper for the GraphicObject) when we would need to take into account the graphic attributes too. This should make the life-cycle much more manageable, but the problem is that there are many many places this needs to be changed.</div>
<div>
I will do the work as much incrementally as possible, with ensuring that the test cover the code and if needed add new tests or extend the existing ones. </div>
<div>
<br /></div>
<div>
Currently almost finished is refactoring of the bitmap table (a list of named bitmaps, mostly used for shape fills or backgrounds) to use XBitmap instead of string based unique ID in the table. For this I needed to change OOXML (oox) and especially the ODF (xmloff) filter, and the document model.</div>
<h2>
<br /></h2>
<h2>
Credits</h2>
<div>
Many thanks to <a href="https://www.documentfoundation.org/">TDF</a> and users that support the foundation by providing donations, to make this work possible. </div>
<div>
<br /></div>
<div>
<br /></div>
<div>
<b><i>To be continued...</i></b></div>
Tomaž Vajngerlhttp://www.blogger.com/profile/17183461757847157603noreply@blogger.com0tag:blogger.com,1999:blog-2640003666891728724.post-77107907975995722282017-05-27T12:48:00.002+02:002017-05-27T12:48:55.431+02:00Pivot charts in LibreOffice: Final part 3It has been a while when I posted an update on pivot charts. In the mean time I finished what was planned and iterated through cycles of needed fixes and polish. In the mean time we branched off the code for LibreOffice 5.4 and the pivot chart implementation is part of that too. If you want to try it out, you can get the LibreOffice 5.4 pre-release on the <a href="https://www.libreoffice.org/download/download/">download</a> page.<br />
<br />
<h2>
Pivot chart field button actions</h2>
Last time I explained about the buttons, but I didn't explain what action is performed when we click on them. The buttons generally have a similar function as in pivot table - to show the pivot table layout and to apply filtering of data. The filtering in the pivot table opens a non-modal windows where you can choose the filtering. For pivot charts I wanted to reuse that, so when clicking on the field button, the request is send from the Chart component back to the Calc, where the same window is shown (shown in Figure 1).<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
</div>
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: left;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhFYdb1wymqyVxDE91A6IpEr_NsrZQvT0jke06qmLRQFCvcHSAybToz05koUCfEbOQ8mSJZiR8HoZqr-Gez-lo5ThN2HXLIHSlWnpgdj8jhY9Kehz3NurnpKU4tni6MdRFyZf7UG23d8URk/s1600/PivotChartFieldFilter.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" height="385" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhFYdb1wymqyVxDE91A6IpEr_NsrZQvT0jke06qmLRQFCvcHSAybToz05koUCfEbOQ8mSJZiR8HoZqr-Gez-lo5ThN2HXLIHSlWnpgdj8jhY9Kehz3NurnpKU4tni6MdRFyZf7UG23d8URk/s400/PivotChartFieldFilter.png" width="400" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">Figure 1: Pivot chart field filter</td></tr>
</tbody></table>
<br /><h2 style="text-align: left;">
Improvements to pivot chart buttons</h2>
<div style="text-align: left;">
In previous post, the pivot chart field buttons were still very basic. Now I improved them, so they show a down arrow, so they look more like they have a pop-up action attached to them. If there is some filtering applied, then the arrow turns blue (similar to the pivot table), so it is easier to see when a field has any filter applied. </div>
<div style="text-align: left;">
For page fields we also show what is filtered: when nothing is filtered "- all -" is shown, when some all filtered, then "- multiple -" is shown and when only one value is not filtered, then we show that value.<br />
<br /></div>
<h2 style="text-align: left;">
ODF support and compatibility</h2>
<div style="text-align: left;">
A pivot chart is useless if we can't save it to a file and later reopen. For this it was needed to extend the ODF format. Luckily, this was relatively easy to do, as the only thing needed is the name of the pivot table that a chart links to (I added "data-pilot-source" attribute to "chart:chart" element). Everything else is already present in the existing import/export code so no additional elements were needed to recreate the exact state that was present when the document was saved. </div>
<div style="text-align: left;">
<br /></div>
<div style="text-align: left;">
A bit related is also copy and paste, which uses the ODF as an intermediate format (copy saves parts to the ODF format and paste loads the format) so things like copy/paste between documents works. A difference here is that we can copy the pivot chart and paste to a different (empty) document, which doesn't have the pivot table. In this case I had to make sure that a normal chart is pasted, which uses the table internal data and not the pivot table. The table internal data is always written with the chart object even if it is not used, just for situations like this (another one is also when we copy from Calc document and paste in Writer document).</div>
<div style="text-align: left;">
<br /></div>
<h2 style="text-align: left;">
Tests</h2>
<div>
It would be really hard for me to implement this properly without tests, as they cemented the behaviour and, if they failed, I knew that probably I made a mistake or I have took a wrong approach to solve the problem. First I added a import / export tests, which just used an existing document to get the data, pivot table and already existing pivot chart. The purpose of these are to test the ODF import and export code. </div>
<div>
Later, I added tests which programatically add data into a sheet and create a pivot table from scratch as a set-up, then create the pivot chart and test various pivot table layouts, and assert what we expect to see in a pivot chart. This approach is better as a document is not needed, and it demonstrates that a pivot chart can be made from scratch with the available API.</div>
<div>
<br /></div>
<h2>
Final demo</h2>
<div>
Finally, I want to show the complete demo of the pivot chart feature:</div>
<div>
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<iframe allowfullscreen="" class="YOUTUBE-iframe-video" data-thumbnail-src="https://i.ytimg.com/vi/txvL1UrsQCw/0.jpg" frameborder="0" height="266" src="https://www.youtube.com/embed/txvL1UrsQCw?feature=player_embedded" width="320"></iframe></div>
<div>
<br /></div>
<div>
<br /></div>
<div>
You can find the video on YouTube at the following URL: <a href="https://youtu.be/txvL1UrsQCw">https://youtu.be/txvL1UrsQCw</a><br />
<br /></div>
<div>
<h2>
Credits</h2>
<div>
Many thanks to Nantes Métropole and Ville de Nantes for making this work possible.</div>
<div>
<br /></div>
<div>
Read more about Nantes deployment <a href="https://conference.libreoffice.org/assets/Conference/Brno/ficheux-migrationof-nantes-metropole.pdf">here</a>.</div>
</div>
Tomaž Vajngerlhttp://www.blogger.com/profile/17183461757847157603noreply@blogger.com0tag:blogger.com,1999:blog-2640003666891728724.post-53596776064316524352017-03-27T10:32:00.003+02:002017-03-27T10:32:49.498+02:00Pivot charts in LibreOffice: Part 2This time I'll present some necessary changes to make pivot charts actually useful and one unique feature that pivot charts have and normal charts don't - field buttons.<br />
<br />
<h2>
Pivot chart creation</h2>
<div>
If you watched the first video, you should notice that I showed the pivot chart was already created from the start. The reason for this was that the functionality to create a new pivot chart from the pivot table wasn't implemented yet. I have fixed this, so it is now possible to create a new pivot chart if you position the cursor on the pivot table, and select from the menu to create a new chart. The chart creation code will detect the pivot table and create a pivot chart instead of a normal chart.</div>
<div>
<br /></div>
<h2>
Pivot chart wizard</h2>
<div>
When we want to create a new chart, we first get the chart creation wizard, where we can select the chart type, define the ranges for labels and data, define data series ranges, and add some additional chart elements like title, subtitle,...</div>
<div>
<br /></div>
<div>
For the pivot chart we get a similar wizard now, where we can select the chart type and additional data. The wizard step to add data ranges and the step to define the data series is however disabled as these steps are not needed when we get the data from the pivot table.<br />
<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj4ImMQJvM-PuXotBAI71hNj84iBrXnBRypNbR-sDFOJwPSZADSehcyKyGD8zwL-OvUsww4Y2dLoStD-3oENs9Rz5p_ksYLJsJFtL34fky0N6PtPPX3-xk4tp-RCA36kew16vdxN4Jmevbm/s1600/Screenshot+from+2017-03-27+10-09-25.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" height="222" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj4ImMQJvM-PuXotBAI71hNj84iBrXnBRypNbR-sDFOJwPSZADSehcyKyGD8zwL-OvUsww4Y2dLoStD-3oENs9Rz5p_ksYLJsJFtL34fky0N6PtPPX3-xk4tp-RCA36kew16vdxN4Jmevbm/s400/Screenshot+from+2017-03-27+10-09-25.png" width="400" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">Pivot chart creation wizard</td></tr>
</tbody></table>
</div>
<div>
<br /></div>
<h2>
Pivot chart buttons</h2>
<div>
This time the biggest change are the pivot chart buttons, which are unique to pivot charts (normal charts don't have them). The purpose of the buttons is to show the layout of the pivot table, so it shows the pivot table fields. On the top it shows the buttons that represent the page fields (if present) and the data fields of the pivot table. At the bottom it shows buttons for row fields next to each other, and in the legend it show the buttons from column fields stacked.<br />
<br /></div>
<div>
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgXSAYH1g0eL9LajDlOwBMIdVJzWP_9_RMU4acuPNDIJLTzgeBrGAs9vnQMrv2w5OwyLU6fT7nsNzDC-RhjO7uorvGGQW2c2-qsIju2Rsh3E7tEUZpFTd5H9LsGybQgztUnhatsTpvgxBq7/s1600/PivotChartButtons.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" height="225" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgXSAYH1g0eL9LajDlOwBMIdVJzWP_9_RMU4acuPNDIJLTzgeBrGAs9vnQMrv2w5OwyLU6fT7nsNzDC-RhjO7uorvGGQW2c2-qsIju2Rsh3E7tEUZpFTd5H9LsGybQgztUnhatsTpvgxBq7/s400/PivotChartButtons.png" width="400" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">Field buttons in a pivot chart</td></tr>
</tbody></table>
<br /></div>
<div>
As they are buttons, there is an action performed when clicking on them, but this is not implemented yet and I'll described this next time in more detail.</div>
<div>
<br /></div>
<div>
From the implementation point of view, the most challenging thing with the buttons was to position them correctly inside the chart as they are part of the chart structure, and to position everything else accordingly (and not breaking the normal charts in the process).<br />
<br /></div>
<div>
<br /></div>
<h2>
Demo</h2>
<div>
This is an updated video of the current state of pivot charts:</div>
<div>
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<iframe allowfullscreen="" class="YOUTUBE-iframe-video" data-thumbnail-src="https://i.ytimg.com/vi/hzl8N9-wpc4/0.jpg" frameborder="0" height="266" src="https://www.youtube.com/embed/hzl8N9-wpc4?feature=player_embedded" width="320"></iframe></div>
<div>
<br /></div>
<div>
You can find the video on YouTube at the following URL: <a href="https://www.youtube.com/watch?v=hzl8N9-wpc4">https://www.youtube.com/watch?v=hzl8N9-wpc4</a></div>
<div>
<br /></div>
<div>
<h2>
Credits</h2>
Again, many thanks to Nantes Métropole and Ville de Nantes for making this work possible.<br />
<br />
Read more about Nantes deployment <a href="https://conference.libreoffice.org/assets/Conference/Brno/ficheux-migrationof-nantes-metropole.pdf">here</a>.<br />
<br />
<b><span style="font-size: large;"><i>To be continued...</i></span></b></div>
Tomaž Vajngerlhttp://www.blogger.com/profile/17183461757847157603noreply@blogger.com0tag:blogger.com,1999:blog-2640003666891728724.post-32234956047272499772017-03-15T12:19:00.001+01:002017-03-15T12:19:24.543+01:00Pivot charts in LibreOffice: Part 1<div dir="ltr">
<h2>
About</h2>
Pivot tables are a powerful tool to reorganise, manipulate and summarise the data set in spreadsheets to get the valuable information from it. To get a quick visual representation of the information, pivot charts can be used. A pivot chart can be created from the output of the pivot tables, and if the pivot table gets changed, so does the pivot chart.<br />
<br />
Support for pivot tables in LibreOffice is available for a long time, but there was no support for pivot charts until now. For the past week I was working on pivot charts in a feature branch (feature/pivotcharts) and I got to a first milestone. Pivot charts will be released in LibreOffice 5.4.<br />
<br />
<h2>
Pivot chart data provider</h2>
<div>
From development point of view, pivot charts are just like normal charts but with a different data provider (source of data), so this was the task with which I started. Normal charts use a data provider which is based around reading from cell ranges, but for pivot charts I created a new data provider, which reads the output data from the pivot table and prepares it for the chart. The data columns are mapped to data series and the data rows become the number of data series in chart (See Figure 1).<br />
<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiqfUmI0sq31HeVBitMSdL1pkAnVFtaIJP-By-uGNY-tQ-BX-Yb96C5wqEWp15IR24mAXgJuQonEijBcejnL9o288mlJTOGM21yLVwf1rIme_O-zBZ4iMlcCni0JFRY_qY4CgUhLnZFYVb2/s1600/PivotTable+to+PivotChart.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" height="400" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiqfUmI0sq31HeVBitMSdL1pkAnVFtaIJP-By-uGNY-tQ-BX-Yb96C5wqEWp15IR24mAXgJuQonEijBcejnL9o288mlJTOGM21yLVwf1rIme_O-zBZ4iMlcCni0JFRY_qY4CgUhLnZFYVb2/s400/PivotTable+to+PivotChart.png" width="361" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">Figure1: Pivot table to pivot chart data mapping</td></tr>
</tbody></table>
Now what is left is naming of each axis and data series in chart. The y-axis categories are mapped to row field names in the pivot table and the data series names, which are shown in the chart are combined names of all column field names of the pivot table.<br />
<br />
Each data point and row or column field name also has an associated number format, which needs to be assign to chart data, otherwise the the number format would not the values correctly as in pivot table (this is especially important with date and time).<br />
<br />
<h2>
Updating a pivot chart</h2>
Once I managed to do the mapping correctly, the pivot chart showed up as expected, but the pivot chart wasn't updated when I update the pivot table. So to solve this, I had to implement a listener of pivot table updates in the pivot chart data provider, and for every update send the signal to chart to update the data again (which it gets from the pivot chart data provider). The whole update procedure sounds like a ping-pong play between components, but it works quite well.<br />
<br />
<h2>
Demo</h2>
In the following video you can see the current status of development:<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<iframe allowfullscreen="" class="YOUTUBE-iframe-video" data-thumbnail-src="https://i9.ytimg.com/vi/xRulORXWhP8/default.jpg?sqp=COzkocYF&rs=AOn4CLDzHy7Gt9jX-WrlxRqTfmU3vpKGRw" frameborder="0" height="266" src="https://www.youtube.com/embed/xRulORXWhP8?feature=player_embedded" width="320"></iframe></div>
<br />
<br />
<h2>
Credits</h2>
One of the real privileges here is working on LibreOffice for a Collabora Productivity customer who funds significant feature work. Many thanks to Nantes Métropole and Ville de Nantes for their investment here, and making this feature available to all LibreOffice users. You can read more about Nantes deployment <a href="https://conference.libreoffice.org/assets/Conference/Brno/ficheux-migrationof-nantes-metropole.pdf">here</a>.<br />
<br />
<b><span style="font-size: large;"><i>To be continued...</i></span></b></div>
</div>
Tomaž Vajngerlhttp://www.blogger.com/profile/17183461757847157603noreply@blogger.com6tag:blogger.com,1999:blog-2640003666891728724.post-66646403497225185542015-04-13T11:56:00.000+02:002015-04-14T01:35:42.999+02:00FOSSASIA 2015On March 12 (wow - that was 1 month ago) I went to FOSSASIA 2015 conference in Singapore. This is my first time visiting Singapore and a conference in Asia. Singapore is located very near the equator so it has quite a constant weather all around the year - which means hot, sometimes humid, and almost daily rainfall. I arrived a day earlier, so I could enjoy one day walking around and exploring Singapore, the very diverse people and food.<br />
<br />
I stayed in Chinatown which is the place that is quite attractive for tourists as it has many bars and shops with Chinese merchandise. Really convenient after a long day to get out for a beer and relax.<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEigFdKRvpKHi2umck-ZhLksUpzWHNOht_rJTYk-qt73aNmRxVwo_5YopLUMbl4g4LcdVsrpBPQ0cNr-hPQ0yTHeQy6e9N6kaLk6KdHyunP4RfZbpQ7QABxcUedTu3R0ky5Wo5I9wfiFKu2Y/s1600/IMG_20150314_080641.jpg" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEigFdKRvpKHi2umck-ZhLksUpzWHNOht_rJTYk-qt73aNmRxVwo_5YopLUMbl4g4LcdVsrpBPQ0cNr-hPQ0yTHeQy6e9N6kaLk6KdHyunP4RfZbpQ7QABxcUedTu3R0ky5Wo5I9wfiFKu2Y/s1600/IMG_20150314_080641.jpg" height="320" width="240" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">Pagoda street, China town in Singapore, early in the morning</td></tr>
</tbody></table>
<br />
<br />
The following day the conference started. On the first day there was only one track, with various and very interesting presentations. For me the most interesting were the talks about systemd, mariadb, Firefox OS and others. I also learned that knitting machines are the next big thing after 3D printing. I'm not a hardware guy but after seeing what some people make I wish I learn more about hardware in my youth.<br />
I was quite impressed by the talk of Dr. Vivian Balakrishnan (Singapore’s Environment Minister) about open data and why it is important for a government (transparency). After the conference there was an organised event at Labrador park, where we enjoyed the barbecue while socializing.<br />
<br />
On the second day of the conference, there were 3 or 4 specialized tracks. I mostly hanged around the "OpenTech" track which still had very diverse talks like: web development, developing methodologies, community, computer vision, etc. Interesting.<br />
<br />
On the last day of the conference I was presenting about LibreOffice on Android (LibreOffice on Android, a
development update). I made a quick introduction to the LibreOffice Viewer which is available in <a href="https://play.google.com/store/apps/details?id=com.collabora.libreoffice&hl=en">Google play</a>, and after that in more detail about editing functionality we are working on currently and is sponsored by <a href="https://www.documentfoundation.org/">TDF</a>.<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgdxaNc_8JkzUxIoezqPVmNr-kMmD7RKnH9hAhrLdrygW2Z1JaLg9LtityGSOQF2efVE4jUcmBVWUKH2-tb9dYlKmL5pFiNqb0hvsyXSh65QUCfdBMWHeCrf-LX-QdtH-whiW2BcnPghqAW/s1600/16649014799_3d26178931_o.jpg" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgdxaNc_8JkzUxIoezqPVmNr-kMmD7RKnH9hAhrLdrygW2Z1JaLg9LtityGSOQF2efVE4jUcmBVWUKH2-tb9dYlKmL5pFiNqb0hvsyXSh65QUCfdBMWHeCrf-LX-QdtH-whiW2BcnPghqAW/s1600/16649014799_3d26178931_o.jpg" height="212" width="320" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">My presentation, picture by Michael Cannon (CC BY 4.0) </td></tr>
</tbody></table>
Slides can be downloaded <a href="http://people.collabora.com/~quikee/LibreOfficeAndroid_FOSSASIA.pdf">here</a>.<br />
<br />
After I finished my talk I visited other tracks I did not visit before but sadly the conference concluded quite soon. I'm looking forward to next year.<br />
<br />
Thanks to FOSSASIA organizers to organize such a wonderful conference and thanks to <a href="https://www.documentfoundation.org/">TDF</a> and <a href="https://libreoffice-from-collabora.com/">Collabora Productivity</a> to make it possible for me to visit the conference.<br />
<br />Tomaž Vajngerlhttp://www.blogger.com/profile/17183461757847157603noreply@blogger.com0tag:blogger.com,1999:blog-2640003666891728724.post-13480152435145168052014-08-19T13:14:00.002+02:002014-08-20T14:29:13.409+02:00LibreOffice on Android #2 - Draw and Impress documents<div dir="ltr">
I am continuing working on LibreOffice on Android and now I had achieved a next milestone - support for Draw and Impress documents.</div>
<div dir="ltr">
<br /></div>
<div dir="ltr">
After some painful debugging experience, where an unavailable service in the Android build prevented the documents to be successfully opened, I finally managed to get a slide rendered on screen:</div>
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgS2TNFMeLRWdqsZIleBhxnhY-nHThoKyl6wvEg4Y7vK_O2PEzRxF4WOKINGd0sc4klGkgdhzOIsVDxpM9SossxW0k-7jrYKtWCV4-CXstdQeDp_YZdp0Mm1LNlBxMLF4ldJ11ZG6X6lpDx/s1600/LOAndroid_20140813_2.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgS2TNFMeLRWdqsZIleBhxnhY-nHThoKyl6wvEg4Y7vK_O2PEzRxF4WOKINGd0sc4klGkgdhzOIsVDxpM9SossxW0k-7jrYKtWCV4-CXstdQeDp_YZdp0Mm1LNlBxMLF4ldJ11ZG6X6lpDx/s1600/LOAndroid_20140813_2.png" height="200" width="320" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">Impress slide rendered in the LibreOffice Android viewer</td></tr>
</tbody></table>
<div dir="ltr">
Draw and Impress documents share most of the code so if you get one working then you most likely get the other for free:</div>
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj8unzpAfil1ptDlvQ0lyslsJFTUweDYlR_QeWy_mB98o-Mii5g1dHS7UVEguZVWLOGzOLDcEtnknff2KWwAldO8FERvc1mNea6wvwWm1jUnSiEPvpmKBIgHveC85gm4Ayi9bELCqIqvJ8R/s1600/Screenshot_2014-08-19-12-18-12.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj8unzpAfil1ptDlvQ0lyslsJFTUweDYlR_QeWy_mB98o-Mii5g1dHS7UVEguZVWLOGzOLDcEtnknff2KWwAldO8FERvc1mNea6wvwWm1jUnSiEPvpmKBIgHveC85gm4Ayi9bELCqIqvJ8R/s1600/Screenshot_2014-08-19-12-18-12.png" height="320" width="200" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">Draw page in LibreOffice Android viewer</td></tr>
</tbody></table>
<div dir="ltr">
Draw and Impress documents differ a little bit from Writer documents, as they are a collection of document parts - slides and pages (and in case of Calc - sheets). To be able to show different parts of the document I implemented a sidebar (using Android's <a href="https://developer.android.com/reference/android/support/v4/widget/DrawerLayout.html">DrawerLayout</a>) which shows all available parts in a list. Clicking on a part in the list re-renders that part on the screen. The sidebar is hidden and can be activated using a left-to-right swipe on the left side of the screen. When activated it looks like this:</div>
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgzC6qquKPIqgEolNhez7cRD6wS-lYpTGGRcOocAiMMxXNyJ74yO1_dPczwj9hBNm71cggmo-R97eh0Jy-ClEGLrIS6RUzOWDOuOOoikla1Mj4M3UyUC3k_M4jq_hdhd3PuBX0orchhu6F9/s1600/LOAndroid_20140813_4.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgzC6qquKPIqgEolNhez7cRD6wS-lYpTGGRcOocAiMMxXNyJ74yO1_dPczwj9hBNm71cggmo-R97eh0Jy-ClEGLrIS6RUzOWDOuOOoikla1Mj4M3UyUC3k_M4jq_hdhd3PuBX0orchhu6F9/s1600/LOAndroid_20140813_4.png" height="200" width="320" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">Sidebar to switch between slides/pages</td></tr>
</tbody></table>
In the future the static icons will be replaced with actual thumbnails of pages.<br />
<div>
<br />
Additional to this I have worked on increasing the performance of tile rendering by auditing the life cycle of tiles and removing corner case bugs. There was a particular bug that removed corner tiles and right after that rendered the same (now missing) tiles for every redraw call (inconsistency in determining which tiles are still valid). Interestingly the impact on the performance was not so big as I thought. I am still not happy with the speed of tile rendering especially when faster scrolling a page.<br />
<br />
All the work so far is summed up in this <a href="http://people.collabora.com/~quikee/LOAndroid_20140813_1.mp4">video</a>.<br />
<br />
<div dir="ltr">
Thanks again goes to <a href="http://www.smoose.nl/">Smoose</a> for continuing the sponsoring of LibreOffice on Android work and my employer <a href="https://libreoffice-from-collabora.com/">Collabora</a> to make this project possible.</div>
<div dir="ltr">
<br /></div>
<div dir="ltr">
Tomaž</div>
</div>
Tomaž Vajngerlhttp://www.blogger.com/profile/17183461757847157603noreply@blogger.com11tag:blogger.com,1999:blog-2640003666891728724.post-11274127095842749752014-07-16T22:13:00.002+02:002014-09-03T16:18:33.306+02:00LibreOffice on AndroidThanks to <a href="http://www.smoose.nl/">Smoose</a>, we are now able to do some real progress with the Android version of LibreOffice. The idea is to first build a LibreOffice document viewer, which is able to display any type of document that is supported by LibreOffice. Afterwards build on that and provide more features and eventually editing. The application itself should be a Android native application and use LibreOffice (interfacing through LibreOfficeKit) to provide tiles of a rendered documents and other needed pieces.<br />
<br />
In the last couple of weeks I have been working on making this plan a reality. The first goal is to prepare the base of the application so I used the Fennec (Firefox for Android) source code. Fennec already solves many of the problems that we would need to solve - especially concerning drawing of tiles, touch handling, scrolling, tools and removed the rest, that will not be needed or needed later in development.<br />
<br />
The calls to Gecko (rendering engine in Firefox) were replaced with our own implementation either in Java or a facade to LibreOfficeKit. By using a mock document tiles (tiles of the document that are part of the application's assets) I was able to make the application work without actually interfacing with LibreOffice yet. With this the application looked something like this:<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEicAMJtxiDStkBt8LoYzdguhFUEpVkLs2DDbDN-iSK2umSsxH_E77hOPNyxwoSDWOob_0cFju-ehiHM4n-qmbXz6z7l-Y3cQ9nriFNYe1NiERWwmWA3OwgPINt5fiRRzxNYhEWyfHeePBiU/s1600/Screenshot_2014-07-16-21-13-42.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEicAMJtxiDStkBt8LoYzdguhFUEpVkLs2DDbDN-iSK2umSsxH_E77hOPNyxwoSDWOob_0cFju-ehiHM4n-qmbXz6z7l-Y3cQ9nriFNYe1NiERWwmWA3OwgPINt5fiRRzxNYhEWyfHeePBiU/s1600/Screenshot_2014-07-16-21-13-42.png" height="320" width="200" /></a></div>
<br />
At this stage the application was able to show a mock document with working scrolling and touch handling. The next big step was integration of LibreOffice and writing a JNI facade to LibreOfficeKit so that a real document rendered by LibreOffice could be show on a screen. With a big help from kendy we managed to integrate LibreOffice and correctly initialize LibreOfficeKit. After that the a real LibreOffice rendered document appeared:<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEga5IRfGTpJVQLKX7yU1AteGqrdNOMrxp2fmXz7MkTd68ZNCaSQj2HaI8fPcD4DeqsBMJOmph8vcXjamxt3qwdpFnzPv3FUt36HpatRxZ4vXrEZo5b5qpiyOogEOezU-BFZTUKTbnOVUB3t/s1600/Screenshot_2014-07-16-20-59-32.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEga5IRfGTpJVQLKX7yU1AteGqrdNOMrxp2fmXz7MkTd68ZNCaSQj2HaI8fPcD4DeqsBMJOmph8vcXjamxt3qwdpFnzPv3FUt36HpatRxZ4vXrEZo5b5qpiyOogEOezU-BFZTUKTbnOVUB3t/s1600/Screenshot_2014-07-16-20-59-32.png" height="320" width="200" /></a></div>
<br />
See application in action video <a href="http://people.collabora.com/~quikee/LOAndroid1.mp4">here</a>.<br />
<br />
<span id="messageContent">Finally some results! There are some issues at the tile borders but this will be eventually resolved. </span>The application uses OpenGL ES 2 for rendering so the user experience is smooth for the most parts (there are still things to optimize). This is the current state of the application but it is still far from complete however a lot of quite difficult technical challenges have been resolved and true development and polishing can now start.<br />
<br />
Next steps are cleaning up and refactor a lot of code, integrate useful parts of previous attempt (LibreOffice4Android), tune tile loading and invalidation (when to load or remove which tile), making parts asynchronous to reduce blocking and improve the user experience, text selection and copy/paste, ...<br />
<br />
I am really excited with what we have achieved and really looking forward to see where we go from here. By the time of LibreOffice 4.4 we should have a working and polished document viewer application ready. Thanks again to <a href="http://www.smoose.nl/">Smoose</a> for funding for the work on this important step!<br />
<br />
Tomaž<br />
<br />
<br />
<br />Tomaž Vajngerlhttp://www.blogger.com/profile/17183461757847157603noreply@blogger.com5