Skip to content

Commit

Permalink
build based on c8b8369
Browse files Browse the repository at this point in the history
  • Loading branch information
Documenter.jl committed Oct 13, 2023
1 parent 1e4158c commit 240de58
Show file tree
Hide file tree
Showing 5 changed files with 7 additions and 7 deletions.
2 changes: 1 addition & 1 deletion dev/devnotes/index.html
Original file line number Diff line number Diff line change
Expand Up @@ -3,4 +3,4 @@
user <--- |state.buffer1| <--- <stream.codec> <--- |state.buffer2| <--- stream

When writing data (`state.mode == :write`):
user ---&gt; |state.buffer1| ---&gt; &lt;stream.codec&gt; ---&gt; |state.buffer2| ---&gt; stream</code></pre><p>In the read mode, a user pull out data from <code>state.buffer1</code> and pre-transcoded data are filled in <code>state.buffer2</code>. In the write mode, a user will push data into <code>state.buffer1</code> and transcoded data are filled in <code>state.buffer2</code>. The default buffer size is 16KiB for each.</p><p><code>State</code> (defined in src/state.jl) has five fields:</p><ul><li><code>mode</code>: current stream mode (<code>&lt;:Symbol</code>)</li><li><code>code</code>: return code of the last codec&#39;s method call (<code>&lt;:Symbol</code>)</li><li><code>error</code>: exception returned by the codec (<code>&lt;:Error</code>)</li><li><code>buffer1</code>: data buffer that is closer to the user (<code>&lt;:Buffer</code>)</li><li><code>buffer2</code>: data buffer that is farther to the user (<code>&lt;:Buffer</code>)</li></ul><p>The <code>mode</code> field may be one of the following value:</p><ul><li><code>:idle</code> : initial and intermediate mode, no buffered data</li><li><code>:read</code> : being ready to read data, data may be buffered</li><li><code>:write</code>: being ready to write data, data may be buffered</li><li><code>:stop</code> : transcoding is stopped, data may be buffered</li><li><code>:close</code>: closed, no buffered data</li><li><code>:panic</code>: an exception has been thrown in codec, data may be buffered but we cannot do anything</li></ul><p>Note that <code>mode=:stop</code> does not mean there is no data available in the stream. This is because transcoded data may be left in the buffer.</p><p>The initial mode is <code>:idle</code> and mode transition happens as shown in the following diagram: <img src="../assets/modes.svg" alt="Mode transition"/></p><p>Modes surrounded by a bold circle are a state in which the transcoding stream has released resources by calling <code>finalize(codec)</code>. The mode transition should happen in the <code>changemode!(stream, newmode)</code> function in src/stream.jl. Trying an undefined transition will thrown an exception.</p><p>A transition happens according to internal or external events of the transcoding stream. The status code and the error object returned by codec methods are internal events, and user&#39;s method calls are external events. For example, calling <code>read(stream)</code> will change the mode from <code>:init</code> to <code>:read</code> and then calling <code>close(stream)</code> will change the mode from <code>:read</code> to <code>:close</code>. When data processing fails in the codec, a codec will return <code>:error</code> and the stream will result in <code>:panic</code>.</p><h2 id="Shared-buffers"><a class="docs-heading-anchor" href="#Shared-buffers">Shared buffers</a><a id="Shared-buffers-1"></a><a class="docs-heading-anchor-permalink" href="#Shared-buffers" title="Permalink"></a></h2><p>Adjacent transcoding streams may share their buffers. This will reduce memory allocation and eliminate data copy between buffers.</p><p><code>readdata!(input::IO, output::Buffer)</code> and <code>writedata!(output::IO, input::Buffer)</code> do the actual work of read/write data from/to the underlying stream. These methods have a special pass for shared buffers.</p><h2 id="Noop-codec"><a class="docs-heading-anchor" href="#Noop-codec"><code>Noop</code> codec</a><a id="Noop-codec-1"></a><a class="docs-heading-anchor-permalink" href="#Noop-codec" title="Permalink"></a></h2><p><code>Noop</code> (<code>NoopStream</code>) is a codec that does <em>nothing</em>. It works as a buffering layer on top of the underlying stream. Since <code>NoopStream</code> does not need to have two distinct buffers, <code>buffer1</code> and <code>buffer2</code> in the <code>State</code> object are shared and some specialized methods are defined for the type. All of these are defined in src/noop.jl.</p></article><nav class="docs-footer"><a class="docs-footer-prevpage" href="../reference/">« Reference</a><div class="flexbox-break"></div><p class="footer-message">Powered by <a href="https://github.com/JuliaDocs/Documenter.jl">Documenter.jl</a> and the <a href="https://julialang.org/">Julia Programming Language</a>.</p></nav></div><div class="modal" id="documenter-settings"><div class="modal-background"></div><div class="modal-card"><header class="modal-card-head"><p class="modal-card-title">Settings</p><button class="delete"></button></header><section class="modal-card-body"><p><label class="label">Theme</label><div class="select"><select id="documenter-themepicker"><option value="documenter-light">documenter-light</option><option value="documenter-dark">documenter-dark</option></select></div></p><hr/><p>This document was generated with <a href="https://github.com/JuliaDocs/Documenter.jl">Documenter.jl</a> version 0.27.25 on <span class="colophon-date" title="Thursday 17 August 2023 23:30">Thursday 17 August 2023</span>. Using Julia version 1.9.2.</p></section><footer class="modal-card-foot"></footer></div></div></div></body></html>
user ---&gt; |state.buffer1| ---&gt; &lt;stream.codec&gt; ---&gt; |state.buffer2| ---&gt; stream</code></pre><p>In the read mode, a user pull out data from <code>state.buffer1</code> and pre-transcoded data are filled in <code>state.buffer2</code>. In the write mode, a user will push data into <code>state.buffer1</code> and transcoded data are filled in <code>state.buffer2</code>. The default buffer size is 16KiB for each.</p><p><code>State</code> (defined in src/state.jl) has five fields:</p><ul><li><code>mode</code>: current stream mode (<code>&lt;:Symbol</code>)</li><li><code>code</code>: return code of the last codec&#39;s method call (<code>&lt;:Symbol</code>)</li><li><code>error</code>: exception returned by the codec (<code>&lt;:Error</code>)</li><li><code>buffer1</code>: data buffer that is closer to the user (<code>&lt;:Buffer</code>)</li><li><code>buffer2</code>: data buffer that is farther to the user (<code>&lt;:Buffer</code>)</li></ul><p>The <code>mode</code> field may be one of the following value:</p><ul><li><code>:idle</code> : initial and intermediate mode, no buffered data</li><li><code>:read</code> : being ready to read data, data may be buffered</li><li><code>:write</code>: being ready to write data, data may be buffered</li><li><code>:stop</code> : transcoding is stopped, data may be buffered</li><li><code>:close</code>: closed, no buffered data</li><li><code>:panic</code>: an exception has been thrown in codec, data may be buffered but we cannot do anything</li></ul><p>Note that <code>mode=:stop</code> does not mean there is no data available in the stream. This is because transcoded data may be left in the buffer.</p><p>The initial mode is <code>:idle</code> and mode transition happens as shown in the following diagram: <img src="../assets/modes.svg" alt="Mode transition"/></p><p>Modes surrounded by a bold circle are a state in which the transcoding stream has released resources by calling <code>finalize(codec)</code>. The mode transition should happen in the <code>changemode!(stream, newmode)</code> function in src/stream.jl. Trying an undefined transition will thrown an exception.</p><p>A transition happens according to internal or external events of the transcoding stream. The status code and the error object returned by codec methods are internal events, and user&#39;s method calls are external events. For example, calling <code>read(stream)</code> will change the mode from <code>:init</code> to <code>:read</code> and then calling <code>close(stream)</code> will change the mode from <code>:read</code> to <code>:close</code>. When data processing fails in the codec, a codec will return <code>:error</code> and the stream will result in <code>:panic</code>.</p><h2 id="Shared-buffers"><a class="docs-heading-anchor" href="#Shared-buffers">Shared buffers</a><a id="Shared-buffers-1"></a><a class="docs-heading-anchor-permalink" href="#Shared-buffers" title="Permalink"></a></h2><p>Adjacent transcoding streams may share their buffers. This will reduce memory allocation and eliminate data copy between buffers.</p><p><code>readdata!(input::IO, output::Buffer)</code> and <code>writedata!(output::IO, input::Buffer)</code> do the actual work of read/write data from/to the underlying stream. These methods have a special pass for shared buffers.</p><h2 id="Noop-codec"><a class="docs-heading-anchor" href="#Noop-codec"><code>Noop</code> codec</a><a id="Noop-codec-1"></a><a class="docs-heading-anchor-permalink" href="#Noop-codec" title="Permalink"></a></h2><p><code>Noop</code> (<code>NoopStream</code>) is a codec that does <em>nothing</em>. It works as a buffering layer on top of the underlying stream. Since <code>NoopStream</code> does not need to have two distinct buffers, <code>buffer1</code> and <code>buffer2</code> in the <code>State</code> object are shared and some specialized methods are defined for the type. All of these are defined in src/noop.jl.</p></article><nav class="docs-footer"><a class="docs-footer-prevpage" href="../reference/">« Reference</a><div class="flexbox-break"></div><p class="footer-message">Powered by <a href="https://github.com/JuliaDocs/Documenter.jl">Documenter.jl</a> and the <a href="https://julialang.org/">Julia Programming Language</a>.</p></nav></div><div class="modal" id="documenter-settings"><div class="modal-background"></div><div class="modal-card"><header class="modal-card-head"><p class="modal-card-title">Settings</p><button class="delete"></button></header><section class="modal-card-body"><p><label class="label">Theme</label><div class="select"><select id="documenter-themepicker"><option value="documenter-light">documenter-light</option><option value="documenter-dark">documenter-dark</option></select></div></p><hr/><p>This document was generated with <a href="https://github.com/JuliaDocs/Documenter.jl">Documenter.jl</a> version 0.27.25 on <span class="colophon-date" title="Friday 13 October 2023 13:58">Friday 13 October 2023</span>. Using Julia version 1.9.3.</p></section><footer class="modal-card-foot"></footer></div></div></div></body></html>
2 changes: 1 addition & 1 deletion dev/examples/index.html
Original file line number Diff line number Diff line change
Expand Up @@ -102,4 +102,4 @@
data1 = read(stream, 8)
TranscodingStreams.unread(stream, data1)
data2 = read(stream, 8)
@assert data1 == data2</code></pre><p>The unread operation is different from the write operation in that the unreaded data are not written to the wrapped stream. The unreaded data are stored in the internal buffer of a transcoding stream.</p><p>Unfortunately, <em>unwrite</em> operation is not provided because there is no way to cancel write operations that are already committed to the wrapped stream.</p></article><nav class="docs-footer"><a class="docs-footer-prevpage" href="../">« Home</a><a class="docs-footer-nextpage" href="../reference/">Reference »</a><div class="flexbox-break"></div><p class="footer-message">Powered by <a href="https://github.com/JuliaDocs/Documenter.jl">Documenter.jl</a> and the <a href="https://julialang.org/">Julia Programming Language</a>.</p></nav></div><div class="modal" id="documenter-settings"><div class="modal-background"></div><div class="modal-card"><header class="modal-card-head"><p class="modal-card-title">Settings</p><button class="delete"></button></header><section class="modal-card-body"><p><label class="label">Theme</label><div class="select"><select id="documenter-themepicker"><option value="documenter-light">documenter-light</option><option value="documenter-dark">documenter-dark</option></select></div></p><hr/><p>This document was generated with <a href="https://github.com/JuliaDocs/Documenter.jl">Documenter.jl</a> version 0.27.25 on <span class="colophon-date" title="Thursday 17 August 2023 23:30">Thursday 17 August 2023</span>. Using Julia version 1.9.2.</p></section><footer class="modal-card-foot"></footer></div></div></div></body></html>
@assert data1 == data2</code></pre><p>The unread operation is different from the write operation in that the unreaded data are not written to the wrapped stream. The unreaded data are stored in the internal buffer of a transcoding stream.</p><p>Unfortunately, <em>unwrite</em> operation is not provided because there is no way to cancel write operations that are already committed to the wrapped stream.</p></article><nav class="docs-footer"><a class="docs-footer-prevpage" href="../">« Home</a><a class="docs-footer-nextpage" href="../reference/">Reference »</a><div class="flexbox-break"></div><p class="footer-message">Powered by <a href="https://github.com/JuliaDocs/Documenter.jl">Documenter.jl</a> and the <a href="https://julialang.org/">Julia Programming Language</a>.</p></nav></div><div class="modal" id="documenter-settings"><div class="modal-background"></div><div class="modal-card"><header class="modal-card-head"><p class="modal-card-title">Settings</p><button class="delete"></button></header><section class="modal-card-body"><p><label class="label">Theme</label><div class="select"><select id="documenter-themepicker"><option value="documenter-light">documenter-light</option><option value="documenter-dark">documenter-dark</option></select></div></p><hr/><p>This document was generated with <a href="https://github.com/JuliaDocs/Documenter.jl">Documenter.jl</a> version 0.27.25 on <span class="colophon-date" title="Friday 13 October 2023 13:58">Friday 13 October 2023</span>. Using Julia version 1.9.3.</p></section><footer class="modal-card-foot"></footer></div></div></div></body></html>
2 changes: 1 addition & 1 deletion dev/index.html
Original file line number Diff line number Diff line change
Expand Up @@ -115,4 +115,4 @@
<td><code>Bzip2DecompressorStream</code></td>
<td>Decompress data in bzip2 (.bz2) format.</td>
</tr>
</table><h2 id="Notes"><a class="docs-heading-anchor" href="#Notes">Notes</a><a id="Notes-1"></a><a class="docs-heading-anchor-permalink" href="#Notes" title="Permalink"></a></h2><h3 id="Wrapped-streams"><a class="docs-heading-anchor" href="#Wrapped-streams">Wrapped streams</a><a id="Wrapped-streams-1"></a><a class="docs-heading-anchor-permalink" href="#Wrapped-streams" title="Permalink"></a></h3><p>The wrapper stream takes care of the wrapped stream. Reading or writing data from or to the wrapped stream outside the management will result in unexpected behaviors. When you close the wrapped stream, you must call the <code>close</code> method of the wrapper stream, which releases allocated resources and closes the wrapped stream.</p><h3 id="Error-handling"><a class="docs-heading-anchor" href="#Error-handling">Error handling</a><a id="Error-handling-1"></a><a class="docs-heading-anchor-permalink" href="#Error-handling" title="Permalink"></a></h3><p>You may encounter an error while processing data with this package. For example, your compressed data may be corrupted or truncated for some reason, and the decompressor cannot recover the original data. In such a case, the codec informs the stream of the error, and the stream goes to an unrecoverable mode. In this mode, the only possible operations are <code>isopen</code> and <code>close</code>. Other operations, such as <code>read</code> or <code>write</code>, will result in an argument error exception. Resources allocated by the codec will be released by the stream, and hence you must not call the finalizer of the codec.</p></article><nav class="docs-footer"><a class="docs-footer-nextpage" href="examples/">Examples »</a><div class="flexbox-break"></div><p class="footer-message">Powered by <a href="https://github.com/JuliaDocs/Documenter.jl">Documenter.jl</a> and the <a href="https://julialang.org/">Julia Programming Language</a>.</p></nav></div><div class="modal" id="documenter-settings"><div class="modal-background"></div><div class="modal-card"><header class="modal-card-head"><p class="modal-card-title">Settings</p><button class="delete"></button></header><section class="modal-card-body"><p><label class="label">Theme</label><div class="select"><select id="documenter-themepicker"><option value="documenter-light">documenter-light</option><option value="documenter-dark">documenter-dark</option></select></div></p><hr/><p>This document was generated with <a href="https://github.com/JuliaDocs/Documenter.jl">Documenter.jl</a> version 0.27.25 on <span class="colophon-date" title="Thursday 17 August 2023 23:30">Thursday 17 August 2023</span>. Using Julia version 1.9.2.</p></section><footer class="modal-card-foot"></footer></div></div></div></body></html>
</table><h2 id="Notes"><a class="docs-heading-anchor" href="#Notes">Notes</a><a id="Notes-1"></a><a class="docs-heading-anchor-permalink" href="#Notes" title="Permalink"></a></h2><h3 id="Wrapped-streams"><a class="docs-heading-anchor" href="#Wrapped-streams">Wrapped streams</a><a id="Wrapped-streams-1"></a><a class="docs-heading-anchor-permalink" href="#Wrapped-streams" title="Permalink"></a></h3><p>The wrapper stream takes care of the wrapped stream. Reading or writing data from or to the wrapped stream outside the management will result in unexpected behaviors. When you close the wrapped stream, you must call the <code>close</code> method of the wrapper stream, which releases allocated resources and closes the wrapped stream.</p><h3 id="Error-handling"><a class="docs-heading-anchor" href="#Error-handling">Error handling</a><a id="Error-handling-1"></a><a class="docs-heading-anchor-permalink" href="#Error-handling" title="Permalink"></a></h3><p>You may encounter an error while processing data with this package. For example, your compressed data may be corrupted or truncated for some reason, and the decompressor cannot recover the original data. In such a case, the codec informs the stream of the error, and the stream goes to an unrecoverable mode. In this mode, the only possible operations are <code>isopen</code> and <code>close</code>. Other operations, such as <code>read</code> or <code>write</code>, will result in an argument error exception. Resources allocated by the codec will be released by the stream, and hence you must not call the finalizer of the codec.</p></article><nav class="docs-footer"><a class="docs-footer-nextpage" href="examples/">Examples »</a><div class="flexbox-break"></div><p class="footer-message">Powered by <a href="https://github.com/JuliaDocs/Documenter.jl">Documenter.jl</a> and the <a href="https://julialang.org/">Julia Programming Language</a>.</p></nav></div><div class="modal" id="documenter-settings"><div class="modal-background"></div><div class="modal-card"><header class="modal-card-head"><p class="modal-card-title">Settings</p><button class="delete"></button></header><section class="modal-card-body"><p><label class="label">Theme</label><div class="select"><select id="documenter-themepicker"><option value="documenter-light">documenter-light</option><option value="documenter-dark">documenter-dark</option></select></div></p><hr/><p>This document was generated with <a href="https://github.com/JuliaDocs/Documenter.jl">Documenter.jl</a> version 0.27.25 on <span class="colophon-date" title="Friday 13 October 2023 13:58">Friday 13 October 2023</span>. Using Julia version 1.9.3.</p></section><footer class="modal-card-foot"></footer></div></div></div></body></html>
Loading

0 comments on commit 240de58

Please sign in to comment.