Skip to content

Commit 6609128

Browse files
committed
Minor cleanup
Add Criterium lib Rename some vars for clarity Update docstring for convert Bump up Clojure to 1.10.3 Remove some unused names
1 parent e521984 commit 6609128

File tree

4 files changed

+110
-55
lines changed

4 files changed

+110
-55
lines changed

README.md

Lines changed: 19 additions & 19 deletions
Original file line numberDiff line numberDiff line change
@@ -8,17 +8,17 @@ Java has a lot of different ways to represent a stream of bytes. Depending on t
88

99
This library is a Rosetta stone for all the byte representations Java has to offer, and gives you the freedom to forget all the APIs you never wanted to know in the first place. Complete documentation can be found [here](http://aleph.io/codox/byte-streams/).
1010

11-
### usage
11+
### Usage
1212

13-
```clj
1413
[byte-streams "0.2.4"]
14+
```clojure
1515
```
1616

17-
### converting types
17+
### Converting types
1818

1919
To convert one byte representation to another, use `byte-streams/convert`:
2020

21-
```clj
21+
```clojure
2222
byte-streams> (convert "abcd" java.nio.ByteBuffer)
2323
#<HeapByteBuffer java.nio.HeapByteBuffer[pos=0 lim=4 cap=4]>
2424
byte-streams> (convert *1 String)
@@ -27,7 +27,7 @@ byte-streams> (convert *1 String)
2727

2828
`(convert data to-type options?)` converts, if possible, the data from its current type to the destination type. This destination type can either be a Java class or a Clojure protocol. However, since there's no direct route from a string to a byte-buffer, under the covers `convert` is doing whatever it takes to get the desired type:
2929

30-
```clj
30+
```clojure
3131
byte-streams> (conversion-path String java.nio.ByteBuffer)
3232
([java.lang.String [B]
3333
[[B java.nio.ByteBuffer])
@@ -37,14 +37,14 @@ While we can't turn a string into a `ByteBuffer`, we can turn a string into a `b
3737

3838
Every type can exist either by itself, or as a sequence. For instance, we can create an `InputStream` representing an infinite number of repeated strings:
3939

40-
```clj
40+
```clojure
4141
byte-stream> (to-input-stream (repeat "hello"))
4242
#<InputStream byte_streams.InputStream@3962a02c>
4343
```
4444

4545
And then we can turn that into a lazy sequence of `ByteBuffers`:
4646

47-
```clj
47+
```clojure
4848
byte-streams> (take 2
4949
(convert *1
5050
(seq-of java.nio.ByteBuffer)
@@ -61,11 +61,11 @@ Notice that we describe a sequence of a type as `(seq-of type)`, and that we've
6161

6262
To create a [Manifold stream](https://github.com/ztellman/manifold), use `(stream-of type)`. To convert a core.async channel, convert it using `manifold.stream/->source`.
6363

64-
### custom conversions
64+
### Custom conversions
6565

6666
While there are conversions defined for all common byte types, this can be extended to other libraries via `byte-streams/def-conversion`:
6767

68-
```clj
68+
```clojure
6969
;; a conversion from byte-buffers to my-byte-representation
7070
(def-conversion [ByteBuffer MyByteRepresentation]
7171
[buf options]
@@ -77,11 +77,11 @@ While there are conversions defined for all common byte types, this can be exten
7777

7878
This mechanism can even be used for types unrelated to byte streams, if you're feeling adventurous.
7979

80-
### transfers
80+
### Transfers
8181

8282
Simple conversions are useful, but sometimes we'll need to do more than just keep the bytes in memory. When you need to write bytes to a file, network socket, or other endpoints, you can use `byte-streams/transfer`.
8383

84-
```clj
84+
```clojure
8585
byte-streams> (def f (File. "/tmp/salutations"))
8686
#'byte-streams/f
8787
byte-streams> (transfer "hello" f {:append? false})
@@ -92,17 +92,17 @@ byte-streams> (to-string f)
9292

9393
`(transfer source sink options?)` allows you pipe anything that can produce bytes into anything that can receive bytes, using the most efficient mechanism available. Custom transfer mechanisms can also be defined:
9494

95-
```clj
95+
```clojure
9696
(def-transfer [InputStream MyByteSink]
9797
[stream sink options]
9898
(send-stream-to-my-sink stream sink))
9999
```
100100

101-
### some utilities
101+
### Some utilities
102102

103103
`byte-streams/print-bytes` will print both hexadecimal and ascii representations of a collection of bytes:
104104

105-
```clj
105+
```clojure
106106
byte-streams> (print-bytes (-> #'print-bytes meta :doc))
107107
50 72 69 6E 74 73 20 6F 75 74 20 74 68 65 20 62 Prints out the b
108108
79 74 65 73 20 69 6E 20 62 6F 74 68 20 68 65 78 ytes in both hex
@@ -113,7 +113,7 @@ byte-streams> (print-bytes (-> #'print-bytes meta :doc))
113113

114114
`(byte-streams/compare-bytes a b)` will return a value which is positive if `a` is lexicographically first, zero if they're equal, and negative otherwise:
115115

116-
```clj
116+
```clojure
117117
byte-streams> (compare-bytes "abc" "abd")
118118
-1
119119
```
@@ -122,7 +122,7 @@ byte-streams> (compare-bytes "abc" "abd")
122122

123123
`byte-streams/conversion-path` returns all the intermediate steps in transforming one type to another, if one exists:
124124

125-
```clj
125+
```clojure
126126
;; each element is a conversion tuple of to/from
127127
byte-streams> (conversion-path java.io.File String)
128128
([java.io.File java.nio.channels.ReadableByteChannel]
@@ -136,19 +136,19 @@ nil
136136

137137
`byte-streams/possible-conversions` returns a list of possible conversion targets for a type.
138138

139-
```clj
139+
```clojure
140140
byte-streams> (possible-conversions String)
141141
(java.lang.String java.io.InputStream java.nio.DirectByteBuffer java.nio.ByteBuffer (seq-of java.nio.ByteBuffer) java.io.Reader java.nio.channels.ReadableByteChannel [B java.lang.CharSequence)
142142
```
143143

144144
`byte-streams/optimized-transfer?` returns true if there is an optimized transfer method for two types:
145145

146-
```clj
146+
```clojure
147147
byte-streams> (optimized-transfer? String java.io.File)
148148
true
149149
```
150150

151-
### license
151+
### License
152152

153153
Copyright © 2014 Zachary Tellman
154154

project.clj

Lines changed: 4 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,5 @@
11
(defproject clj-commons/byte-streams (or (System/getenv "version") "0.2.5-alpha")
2-
:description "A simple way to handle the menagerie of Java byte represenations."
2+
:description "A simple way to handle the menagerie of Java byte representations."
33
:license {:name "MIT License"
44
:url "http://opensource.org/licenses/MIT"}
55

@@ -10,10 +10,11 @@
1010
:dependencies [[primitive-math "0.1.6"]
1111
[clj-tuple "0.2.2"]
1212
[manifold "0.1.9"]]
13-
:profiles {:dev {:dependencies [[org.clojure/clojure "1.10.1"]
13+
:profiles {:dev {:dependencies [[org.clojure/clojure "1.10.3"]
1414
[org.clojure/test.check "1.1.0"]
1515
[rhizome "0.2.9"]
16-
[codox-md "0.2.0" :exclusions [org.clojure/clojure]]]}
16+
[codox-md "0.2.0" :exclusions [org.clojure/clojure]]
17+
[criterium "0.4.6"]]}
1718
:ci {:javac-options ["-target" "1.8" "-source" "1.8"]
1819
:dependencies [[org.clojure/clojure "1.10.1"]
1920
[org.clojure/test.check "1.1.0"]

src/byte_streams.clj

Lines changed: 85 additions & 27 deletions
Original file line numberDiff line numberDiff line change
@@ -57,16 +57,16 @@
5757
(defonce inverse-conversions (atom (g/conversion-graph)))
5858
(defonce src->dst->transfer (atom nil))
5959

60-
(def ^:private ^:const byte-array (class (Utils/byteArray 0)))
60+
(def ^:private ^:const byte-array-type (class (Utils/byteArray 0)))
6161

6262
(defn seq-of [x]
63-
(g/type 'seq (if (identical? bytes x) byte-array x)))
63+
(g/type 'seq (if (identical? bytes x) byte-array-type x)))
6464

6565
(defn stream-of [x]
66-
(g/type 'stream (if (identical? bytes x) byte-array x)))
66+
(g/type 'stream (if (identical? bytes x) byte-array-type x)))
6767

6868
(defn vector-of [x]
69-
(g/type 'vector (if (identical? bytes x) byte-array x)))
69+
(g/type 'vector (if (identical? bytes x) byte-array-type x)))
7070

7171
(defn type-descriptor
7272
"Returns a descriptor of the type of the given instance."
@@ -77,7 +77,7 @@
7777
(g/type ::nil)
7878

7979
(identical? bytes x)
80-
(g/type byte-array)
80+
(g/type byte-array-type)
8181

8282
(vector? x)
8383
(vector-of (.type ^Type (type-descriptor (first x))))
@@ -97,7 +97,7 @@
9797
x
9898

9999
(or (= 'bytes x) (= bytes x))
100-
(g/type byte-array)
100+
(g/type byte-array-type)
101101

102102
:else
103103
(g/type (eval x))))
@@ -168,18 +168,23 @@
168168

169169
(defn convert
170170
"Converts `x`, if possible, into type `dst`, which can be either a class or protocol. If no such conversion
171-
is possible, an IllegalArgumentException is thrown. If `x` is a stream, then the `src` type must be explicitly specified.
171+
is possible, an IllegalArgumentException is thrown. If `x` is a stream, then the `src` type must be explicitly
172+
specified.
172173
173174
`options` is a map, whose available settings depend on what sort of transform is being performed:
174175
175-
`chunk-size` - if a stream is being transformed into a sequence of discrete chunks, `:chunk-size` describes the
176-
size of the chunks, which default to 4096 bytes.
176+
`chunk-size` - if a stream is being transformed into a sequence of discrete chunks, `:chunk-size` describes the
177+
size of the chunks, which default to 4096 bytes.
177178
178-
`encoding` - if a string is being encoded or decoded, `:encoding` describes the charset that is used, which
179-
defaults to 'UTF-8'
179+
`encoding` - if a string is being encoded or decoded, `:encoding` describes the charset that is used, which
180+
defaults to 'UTF-8'
181+
182+
`direct?` - if a byte-buffer is being allocated, `:direct?` describes whether it should be a direct buffer,
183+
defaulting to false
184+
`source-type` - overrides input type detection, required to convert a stream
180185
181-
`direct?` - if a byte-buffer is being allocated, `:direct?` describes whether it should be a direct buffer,
182-
defaulting to false"
186+
(NB: if you need to convert a stream to a seq, or vice versa, but not the underlying byte type, you want
187+
Manifold's `stream->seq` and `->source` instead)"
183188
([x dst]
184189
(convert x dst nil))
185190
([x dst options]
@@ -188,7 +193,7 @@
188193
^Type
189194
src (g/type
190195
(or source-type
191-
(type-descriptor x)))
196+
(type-descriptor x)))
192197
wrapper (.wrapper src)]
193198

194199
(cond
@@ -465,7 +470,7 @@
465470
(Channels/newChannel input-stream))
466471

467472
;; string => byte-array
468-
(def-conversion ^{:cost 2} [String byte-array]
473+
(def-conversion ^{:cost 2} [String byte-array-type]
469474
[s {:keys [encoding] :or {encoding "UTF-8"}}]
470475
(.getBytes s ^String (name encoding)))
471476

@@ -507,7 +512,7 @@
507512
(.close out))))
508513
in))
509514

510-
(def-conversion ^{:cost 1.5} [InputStream byte-array]
515+
(def-conversion ^{:cost 1.5} [InputStream byte-array-type]
511516
[in options]
512517
(let [out (ByteArrayOutputStream. (p/max 64 (.available in)))
513518
buf (Utils/byteArray 16384)]
@@ -519,11 +524,11 @@
519524
(.toByteArray out)))
520525

521526
#_(let [ary (Utils/byteArray 0)]
522-
(def-conversion ^{:cost 0} [::nil byte-array]
527+
(def-conversion ^{:cost 0} [::nil byte-array-type]
523528
[src options]
524529
ary))
525530

526-
(def-conversion ^{:cost 2} [#'proto/ByteSource byte-array]
531+
(def-conversion ^{:cost 2} [#'proto/ByteSource byte-array-type]
527532
[src options]
528533
(let [os (ByteArrayOutputStream.)]
529534
(transfer src os)
@@ -673,7 +678,7 @@
673678
OutputStream
674679
(send-bytes! [this b _]
675680
(let [^OutputStream os this]
676-
(.write os ^bytes (convert b byte-array))))
681+
(.write os ^bytes (convert b byte-array-type))))
677682

678683
WritableByteChannel
679684
(send-bytes! [this b _]
@@ -766,7 +771,7 @@
766771
([x options]
767772
(condp instance? x
768773
ByteBuffer x
769-
byte-array (ByteBuffer/wrap x)
774+
byte-array-type (ByteBuffer/wrap x)
770775
String (ByteBuffer/wrap (.getBytes ^String x (name (get options :encoding "UTF-8"))))
771776
(convert x ByteBuffer options))))
772777

@@ -783,24 +788,24 @@
783788
(to-byte-array x nil))
784789
([x options]
785790
(condp instance? x
786-
byte-array x
791+
byte-array-type x
787792
String (.getBytes ^String x (name (get options :encoding "UTF-8")))
788-
(convert x byte-array options))))
793+
(convert x byte-array-type options))))
789794

790795
(defn to-byte-arrays
791796
"Converts the object to a byte-array."
792797
([x]
793798
(to-byte-array x nil))
794799
([x options]
795-
(convert x (seq-of byte-array) options)))
800+
(convert x (seq-of byte-array-type) options)))
796801

797802
(defn ^InputStream to-input-stream
798803
"Converts the object to a `java.io.InputStream`."
799804
([x]
800805
(to-input-stream x nil))
801806
([x options]
802807
(condp instance? x
803-
byte-array (ByteArrayInputStream. x)
808+
byte-array-type (ByteArrayInputStream. x)
804809
ByteBuffer (ByteBufferInputStream. x)
805810
(convert x InputStream options))))
806811

@@ -843,7 +848,7 @@
843848
(let [encoding (get options :encoding "UTF-8")]
844849
(condp instance? x
845850
String x
846-
byte-array (String. ^"[B" x ^String (name encoding))
851+
byte-array-type (String. ^"[B" x ^String (name encoding))
847852
(convert x String options)))))
848853

849854
(defn to-reader
@@ -934,11 +939,11 @@
934939
^long [a b]
935940
(if (and
936941
(or
937-
(instance? byte-array a)
942+
(instance? byte-array-type a)
938943
(instance? ByteBuffer a)
939944
(instance? String a))
940945
(or
941-
(instance? byte-array b)
946+
(instance? byte-array-type b)
942947
(instance? ByteBuffer b)
943948
(instance? String b)))
944949
(cmp-bufs (to-byte-buffer a) (to-byte-buffer b))
@@ -960,3 +965,56 @@
960965
"Returns true if the two byte streams are equivalent."
961966
[a b]
962967
(p/== 0 (compare-bytes a b)))
968+
969+
970+
971+
(comment
972+
(require '[manifold.stream :as ms])
973+
974+
(def content
975+
(doto (ms/stream)
976+
(ms/put! (clojure.core/byte-array 5))
977+
(ms/close!)))
978+
979+
(conversion-path (stream-of bytes) (seq-of bytes)) ; => ([(stream-of [B) (seq-of [B)])
980+
981+
(convert content (seq-of bytes)) ; doesn't work
982+
(= content (convert content (seq-of bytes)))
983+
984+
(convert content (seq-of bytes) {:source-type (stream-of bytes)}) ; works
985+
(= content (convert content (seq-of bytes) {:source-type (stream-of bytes)}))
986+
987+
(convert content (seq-of bytes) {:source-type (stream-of nil)}) ; doesn't work
988+
(= content (convert content (seq-of bytes) {:source-type (stream-of nil)}))
989+
990+
(convert content (seq-of String)) ; also doesn't work
991+
(= content (convert content (seq-of String)))
992+
993+
(convert content (seq-of java.nio.ByteBuffer)) ; this works
994+
(= content (convert content (seq-of java.nio.ByteBuffer)))
995+
996+
997+
998+
(let [content (doto (ms/stream)
999+
(ms/put! (clojure.core/byte-array 5))
1000+
(ms/close!))]
1001+
(convert content (seq-of bytes)) ; no
1002+
#_ (convert content (seq-of bytes) {:source-type (stream-of bytes)}) ; yes
1003+
#_(convert content (seq-of bytes) {:source-type (stream-of nil)}) ; no
1004+
)
1005+
1006+
1007+
(let [content (doto (ms/stream)
1008+
(ms/put! (clojure.core/byte-array 5))
1009+
(ms/close!))]
1010+
(convert content (seq-of String)) ; also doesn't work
1011+
(= content (convert content (seq-of String))))
1012+
1013+
(let [content (doto (ms/stream)
1014+
(ms/put! (clojure.core/byte-array 5))
1015+
(ms/close!))]
1016+
(convert content (seq-of java.nio.ByteBuffer)) ; this works
1017+
(= content (convert content (seq-of java.nio.ByteBuffer))))
1018+
1019+
1020+
)

0 commit comments

Comments
 (0)