[Paraview] VTK XML binary file format

Discussion:

Peter Bastian

2006-01-11 17:13:26 UTC

Hi all,

I am trying to write VTK XML unstructured grid files (.vtu) by myself
(i.e. without using vtkXMLUnstructuredGridWriter). Ascii works fine but
binary makes problems. I followed the instructions given previously on
this list, i.e. I use inline DataArrays, put the size of the data in
front of the block and encode everything base64.

Now there are two problems:

1) The "offsets" array causes an error when reading:

***@cray:~/FE/dune-dd$ paraview
ErrorMessage
# Error or warning: There was a VTK Error in file:
/home/amy/ParaViewReleaseRoot/paraview-2.4.1/VTK/IO/vtkXMLUnstructuredDataReader.cxx
(592)
vtkXMLUnstructuredGridReader (0x8dfacb0): Cannot read cell connectivity
from Cells in piece 0 because the "offsets" array is not monotonically
increasing or starts with a value less than 1.
ErrorMessage end

When I write only the offsets array in ascii, the file is read without
error message. This is curious because all the writing is done by one
class in my code. I also checked the encoded binary data and it looks OK.

2) Loading the file with all in binary except the "offsets" array, the
point data as well as the cell data and the point positions are not correct.

Now my question:

Is there some further undocumented magic (like the size in bytes at the
beginning) that I have to consider?

I would prefer to write VTK files without linking in VTK itself.
In the attachment I append binary and ascii versions of my file.

Thanks for any response.

-- Peter
------------------------------------------------------------------
Peter Bastian, IWR,Uni Heidelberg, INF 348,R 020, 69120 Heidelberg
email: ***@iwr.uni-heidelberg.de Tel: +49 6221 54 4984
WWW: http://www.iwr.uni-heidelberg.de/~Peter.Bastian Fax: ... 8860

-------------- next part --------------
A non-text attachment was scrubbed...
Name: test.ascii.vtu
Type: text/xml
Size: 1565 bytes
Desc: not available
Url : http://public.kitware.com/pipermail/paraview/attachments/20060111/f7451859/test.ascii.bin
-------------- next part --------------
A non-text attachment was scrubbed...
Name: test.binary.vtu
Type: text/xml
Size: 1950 bytes
Desc: not available
Url : http://public.kitware.com/pipermail/paraview/attachments/20060111/f7451859/test.binary.bin

Dominik Szczerba

2006-01-11 17:16:41 UTC

Permalink

is there any particular reason you write xml files yourself?
I take an easier way out writing binary legacy files and post-converting
them once written.
PS/ and I do confirm strange things happening with xml files
--dsz

Post by Peter Bastian
Hi all,
I am trying to write VTK XML unstructured grid files (.vtu) by myself
(i.e. without using vtkXMLUnstructuredGridWriter). Ascii works fine but
binary makes problems. I followed the instructions given previously on
this list, i.e. I use inline DataArrays, put the size of the data in
front of the block and encode everything base64.
ErrorMessage
/home/amy/ParaViewReleaseRoot/paraview-2.4.1/VTK/IO/vtkXMLUnstructuredDataReader.cxx
(592)
vtkXMLUnstructuredGridReader (0x8dfacb0): Cannot read cell connectivity
from Cells in piece 0 because the "offsets" array is not monotonically
increasing or starts with a value less than 1.
ErrorMessage end
When I write only the offsets array in ascii, the file is read without
error message. This is curious because all the writing is done by one
class in my code. I also checked the encoded binary data and it looks OK.
2) Loading the file with all in binary except the "offsets" array, the
point data as well as the cell data and the point positions are not correct.
Is there some further undocumented magic (like the size in bytes at the
beginning) that I have to consider?
I would prefer to write VTK files without linking in VTK itself.
In the attachment I append binary and ascii versions of my file.
Thanks for any response.
-- Peter
------------------------------------------------------------------
Peter Bastian, IWR,Uni Heidelberg, INF 348,R 020, 69120 Heidelberg
WWW: http://www.iwr.uni-heidelberg.de/~Peter.Bastian Fax: ... 8860
------------------------------------------------------------------------
<?xml version="1.0"?>
<VTKFile type="UnstructuredGrid" version="0.1" byte_order="LittleEndian">
<UnstructuredGrid>
<Piece NumberOfPoints="25" NumberOfCells="16">
<PointData Scalars="exact">
<DataArray type="Float32" Name="exact" format="ascii">
0 0 0 0.0625 0 0.125 0 0.1875 0 0.25 0 0.125
0.25 0.375 0.5 0 0.1875 0.375 0.5625 0.75 0 0.25 0.5 0.75
1
</DataArray>
</PointData>
<CellData Scalars="exact">
<DataArray type="Float32" Name="exact" format="ascii">
0.015625 0.046875 0.078125 0.109375 0.046875 0.140625 0.234375 0.328125 0.078125 0.234375 0.390625 0.546875
0.109375 0.328125 0.546875 0.765625
</DataArray>
</CellData>
<Points>
<DataArray type="Float32" Name="Coordinates" NumberOfComponents="3" format="ascii">
0 0 0 0.25 0 0 0 0.25 0 0.25 0.25 0
0.5 0 0 0.5 0.25 0 0.75 0 0 0.75 0.25 0
1 0 0 1 0.25 0 0 0.5 0 0.25 0.5 0
0.5 0.5 0 0.75 0.5 0 1 0.5 0 0 0.75 0
0.25 0.75 0 0.5 0.75 0 0.75 0.75 0 1 0.75 0
0 1 0 0.25 1 0 0.5 1 0 0.75 1 0
1 1 0
</DataArray>
</Points>
<Cells>
<DataArray type="Int32" Name="connectivity" format="ascii">
0 1 3 2 1 4 5 3 4 6 7 5
6 8 9 7 2 3 11 10 3 5 12 11
5 7 13 12 7 9 14 13 10 11 16 15
11 12 17 16 12 13 18 17 13 14 19 18
15 16 21 20 16 17 22 21 17 18 23 22
18 19 24 23
</DataArray>
<DataArray type="Int32" Name="offsets" format="ascii">
4 8 12 16 20 24 28 32 36 40 44 48
52 56 60 64
</DataArray>
<DataArray type="UInt8" Name="types" format="ascii">
9 9 9 9 9 9 9 9 9 9 9 9
9 9 9 9
</DataArray>
</Cells>
</Piece>
</UnstructuredGrid>
</VTKFile>
------------------------------------------------------------------------
<?xml version="1.0"?>
<VTKFile type="UnstructuredGrid" version="0.1" byte_order="LittleEndian">
<UnstructuredGrid>
<Piece NumberOfPoints="25" NumberOfCells="16">
<PointData Scalars="exact">
<DataArray type="Float32" Name="exact" format="binary">
ZAAAAAAAAAAAAAAAAAAAAAAAgD0AAAAAAAAAPgAAAAAAAEA+AAAAAAAAgD4AAAAAAAAAPgAA
gD4AAMA+AAAAPwAAAAAAAEA+AADAPgAAED8AAEA/AAAAAAAAgD4AAAA/AABAPwAAgD8=
</DataArray>
</PointData>
<CellData Scalars="exact">
<DataArray type="Float32" Name="exact" format="binary">
QAAAAAAAgDwAAEA9AACgPQAA4D0AAEA9AAAQPgAAcD4AAKg+AACgPQAAcD4AAMg+AAAMPwAA
4D0AAKg+AAAMPwAARD8=
</DataArray>
</CellData>
<Points>
<DataArray type="Float32" Name="Coordinates" NumberOfComponents="3" format="binary">
LAEAAAAAAAAAAAAAAAAAAAAAgD4AAAAAAAAAAAAAAAAAAIA+AAAAAAAAgD4AAIA+AAAAAAAA
AD8AAAAAAAAAAAAAAD8AAIA+AAAAAAAAQD8AAAAAAAAAAAAAQD8AAIA+AAAAAAAAgD8AAAAA
AAAAAAAAgD8AAIA+AAAAAAAAAAAAAAA/AAAAAAAAgD4AAAA/AAAAAAAAAD8AAAA/AAAAAAAA
QD8AAAA/AAAAAAAAgD8AAAA/AAAAAAAAAAAAAEA/AAAAAAAAgD4AAEA/AAAAAAAAAD8AAEA/
AAAAAAAAQD8AAEA/AAAAAAAAgD8AAEA/AAAAAAAAAAAAAIA/AAAAAAAAgD4AAIA/AAAAAAAA
AD8AAIA/AAAAAAAAQD8AAIA/AAAAAAAAgD8AAIA/AAAAAA==
</DataArray>
</Points>
<Cells>
<DataArray type="Int32" Name="connectivity" format="binary">
AAEAAAAAAAABAAAAAwAAAAIAAAABAAAABAAAAAUAAAADAAAABAAAAAYAAAAHAAAABQAAAAYA
AAAIAAAACQAAAAcAAAACAAAAAwAAAAsAAAAKAAAAAwAAAAUAAAAMAAAACwAAAAUAAAAHAAAA
DQAAAAwAAAAHAAAACQAAAA4AAAANAAAACgAAAAsAAAAQAAAADwAAAAsAAAAMAAAAEQAAABAA
AAAMAAAADQAAABIAAAARAAAADQAAAA4AAAATAAAAEgAAAA8AAAAQAAAAFQAAABQAAAAQAAAA
EQAAABYAAAAVAAAAEQAAABIAAAAXAAAAFgAAABIAAAATAAAAGAAAABcAAAA=
</DataArray>
<DataArray type="Int32" Name="offsets" format="binary">
QAAAAAQAAAAIAAAADAAAABAAAAAUAAAAGAAAABwAAAAgAAAAJAAAACgAAAAsAAAAMAAAADQA
AAA4AAAAPAAAAEAAAAA=
</DataArray>
<DataArray type="UInt8" Name="types" format="binary">
EAAAAAkJCQkJCQkJCQkJCQkJCQk=
</DataArray>
</Cells>
</Piece>
</UnstructuredGrid>
</VTKFile>
------------------------------------------------------------------------
_______________________________________________
ParaView mailing list
http://www.paraview.org/mailman/listinfo/paraview

--
Dominik Szczerba, Dr.
ETZ C110 CO-ME/BIWI CH-8092 Zurich
http://www.vision.ee.ethz.ch/~domi

Peter Bastian

2006-01-11 18:16:22 UTC

Permalink

Thanks for answering so fast!

Post by Dominik Szczerba
is there any particular reason you write xml files yourself?

I understood that I have to use XML to do parallel visualization.
Furthermore I don't want to install VTK on every machine where I compute
my results, therefore it should be "stand-alone".

Post by Dominik Szczerba
I take an easier way out writing binary legacy files and post-converting
them once written.

Hmmm, that adds another conversion step. Not good with TBytes of data.
You do that in parallel?

Post by Dominik Szczerba
PS/ and I do confirm strange things happening with xml files

:-(

-- Peter

Post by Dominik Szczerba
--dsz

--
------------------------------------------------------------------
Peter Bastian, IWR,Uni Heidelberg, INF 348,R 020, 69120 Heidelberg
email: ***@iwr.uni-heidelberg.de Tel: +49 6221 54 4984
WWW: http://www.iwr.uni-heidelberg.de/~Peter.Bastian Fax: ... 8860

Dominik Szczerba

2006-01-11 18:26:04 UTC

Permalink

no, I am not concerned about parallel performance (yet).
still might be easier to install vtk on each and every machine you work
on (esp. for the same system).
if you want to bite the xml file format sharpen your teeth well.
(and post the results)
good luck
--dsz

Post by Peter Bastian
Thanks for answering so fast!

Post by Dominik Szczerba
is there any particular reason you write xml files yourself?

I understood that I have to use XML to do parallel visualization.
Furthermore I don't want to install VTK on every machine where I compute
my results, therefore it should be "stand-alone".

Post by Dominik Szczerba
I take an easier way out writing binary legacy files and post-converting
them once written.

Hmmm, that adds another conversion step. Not good with TBytes of data.
You do that in parallel?

Post by Dominik Szczerba
PS/ and I do confirm strange things happening with xml files

:-(
-- Peter

Post by Dominik Szczerba
--dsz

--
Dominik Szczerba, Dr.
ETZ C110 CO-ME/BIWI CH-8092 Zurich
http://www.vision.ee.ethz.ch/~domi

Olivier Magotte

2006-01-11 17:35:56 UTC

Permalink

I suggest to encode the data array size (4 bytes) first and then encode
the array (two '=' character must appear after the first 6 characters).
It works for me.

Useful information can also be found here:
http://public.kitware.com/pipermail/paraview/2005-April/001391.html

Olivier

--
--------------------------------------------------------------------------------
Ir. Olivier Magotte | CENAERO
Research Scientist - Virtual Manufacturing | www.cenaero.be
--------------------------------------------------------------------------------

Peter Bastian

2006-01-11 19:15:34 UTC

Permalink

Post by Olivier Magotte

I suggest to encode the data array size (4 bytes) first and then encode
the array (two '=' character must appear after the first 6 characters).
It works for me.
http://public.kitware.com/pipermail/paraview/2005-April/001391.html
Olivier

Strange. This does not work at all. Does not give an error message, but
the mesh then has 0 cells and 0 vertices. Also, it is in contrast to the
reference you have sent. I use uncompressed data.

Thanks anyway.

-- Peter
------------------------------------------------------------------
Peter Bastian, IWR,Uni Heidelberg, INF 348,R 020, 69120 Heidelberg
email: ***@iwr.uni-heidelberg.de Tel: +49 6221 54 4984
WWW: http://www.iwr.uni-heidelberg.de/~Peter.Bastian Fax: ... 8860

Thierry Dubuis

2006-01-11 18:38:16 UTC

Permalink

Hello Peter,

Sorry, I don't read the binary Base64 encode directly, but after debug
Paraview
I saw the twelve firsts offsets [0-11] are read Ok but the following
offsets (12,13,14,15) have the value -842203084

You must have a problem in your write function.

I hope this informations could help you to find your problem.

Best regards,
Thierry Dubuis