Share
 Twitter
 Feeds
 Contribute

clojure.pprint/cl-format is slow

on June 5, 2016, 12:51 pm

Had a serious performance problem last week. After some digging we narrowed it down to a call to cl-format.

Some test code to demonstrate the discrepancy:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
;; Format a string, an integer, a conditional and fancy plural thing
(defn cl-format-test [] (clojure.pprint/cl-format nil "
    [email protected][(Id: ~D)~] ~:[foo~;bar~] [~D second~:P ago]>"
                                      (java.util.UUID/randomUUID)
                                      (rand 1)
                                      (> (rand 2))
                                      (rand 100)))
;; Same inputs, just concat them and return the string
(defn str-test [] (str (java.util.UUID/randomUUID) " "
                            (rand 1) " "
                            (> (rand 2)) " "
                            (rand 100)))
;; Just format a string
(defn cl-format-simple-test [] (clojure.pprint/cl-format nil "~S" (java.util.UUID/randomUUID)))
;; Just return the string
(defn str-simple-test [] (str (java.util.UUID/randomUUID)))

(defn test-func [f tests] (time (doseq [_ (range 0 tests)] (f))))

(test-func cl-format-test 1000)
(test-func str-test 1000)
(test-func cl-format-simple-test 1000)
(test-func str-simple-test 1000)

With the following results. For the simple case, cl-format is 10 times slower than a simple string return. For more complicated format strings, closer to 100 times slower:

(test-func cl-format-test 1000)
"Elapsed time: 328.033183 msecs"
(test-func str-test 1000)
"Elapsed time: 5.939625 msecs"
(test-func cl-format-simple-test 1000)
"Elapsed time: 34.2998 msecs"
(test-func str-simple-test 1000)
"Elapsed time: 3.515013 msecs"

Having a quick look at the implementation of cl-format I immediately noticed a compile-format function which seems like something that might help out with performance for many calls to a cl-format with the same format string:

Compiles format-str into a compiled format which can be used as an argument to cl-format just like a plain format string. Use this function for improved performance when you're using the same format string repeatedly

It also seems like cl-format will check if it's format string is already compiled and skip compilation if that is the case:

(if (string? format-in) (compile-format format-in) format-in)

Oddly however, compile-format is not public, so I can't use it. So I am left a little confused. I am going to do some more digging tomorrow to solve this mystery as I am sure I am missing something here.

Not sure if pre-compilation will help solve the horrible performance you get from cl-format, but I am guessing it might help.

Update

After some discussion with my colleagues, it seems I haven't missed anything and we think this is a bug.

Permalink - Comments - Tags: Development,Clojure

Decoding Polylines from Google Maps Direction API with Clojure

on January 17, 2016, 12:56 pm

My first crack at porting some imperative code to Clojure goodness.

I needed some code to turn the polyline points encoding you get back from Google Directions API. This stuff: 

"overview_polyline" : {
            "points" : "[email protected]|FsBpG_C_BqA}@mGeEwH_FsDcCaGwDkNiJsJmG_SoMyByAzA}[email protected]{[email protected]@@[email protected]}@{[email protected][email protected]]^[email protected]|AcC|[email protected]@[email protected]`AU~A_ChH_D|[email protected]|@[email protected]@[email protected]@[email protected]@Z^X^[email protected][email protected]`@lA^`@[email protected]@[email protected]@`@[email protected]@[email protected]^[email protected]@[email protected]][email protected]@}@[email protected]@qC][email protected]@@eAT[L[[email protected]@SZQ^[email protected]`XoAbEmB~GqAvDkDzIsFvNcAvCcFrRkBnHcE`[email protected][email protected]@hA}@[email protected]@[email protected]|@[email protected]@[email protected]`@aB{[email protected]@yAmE`A}[email protected][email protected]{BhCoBtCuJrOkAxAkBzCeH~KkAlBeMrR{I|M}CtEyBjD}EpHmLbQ{[email protected]@[email protected]@HiABuKT_BEiBUaEkAcA][email protected]@][email protected]@[email protected][email protected]@^`@[email protected]@[email protected]@[email protected]@[email protected]@[email protected]@[email protected]@G^[email protected]@[email protected]@Z`[email protected]@Q"

I found Jeffrey Sambells' Java code here that does the job. I am just getting my teeth into writing Clojure for my day job at GoCatch, so I need a Clojure version. This is my first attempt. It's midnight and I haven't had a chance to check the line ends up on a map correctly, but it looks pretty good to me:

;; Port of the Java code to decode google polylines that I found here -> http://jeffreysambells.com/2010/05/27/decoding-polylines-from-google-maps-direction-api-with-java
(defn decode-next-result [encoded]
  ;; keep scanning through encoded till b>=0x20
  ;; returns the next latitude/longitude increment
  (loop [[current & rest] encoded shift 0 result 0]
    (let [b       (- (int current) 63)
          result  (bit-or result (bit-shift-left (bit-and b 0x1f) shift))
          shift   (+ shift 5)] 
      (if (>= b 0x20) 
        (do 
          ;; if we are encoding the next result then we 
          ;; must have more characters to scan
          (assert rest)
          ;; keep looking for our next result
          (recur rest shift result))
        ;; we found our next result 
        (let [return-value (if (not= (bit-and result 1) 0)
                             (bit-not (bit-shift-right result 1))
                             (bit-shift-right result 1))]
          [return-value rest])))))

(defn lat-lng-double [lat-lng-int]
  (/ lat-lng-int 1E5))

(defn path-for-encoded-polyline [encoded] 
  (loop [rest encoded lat 0 lng 0 results []]
    (if rest
      ;; if there is anthing in the encoded array
      ;; we should have two more results at least 
      (let [next-result       (decode-next-result rest)
            new-rest          (second next-result)
            next-lat-result   (+ lat (first next-result))
            next-result       (decode-next-result new-rest )
            new-rest          (second next-result)
            next-lng-result   (+ lng (first next-result))]
          ;; add our lat lng result to the results
          (recur new-rest next-lat-result next-lng-result (conj 
                                                            results 
                                                            {:latitude (lat-lng-double next-lat-result) 
                                                             :longitude (lat-lng-double next-lng-result)})))
      ;; we are done, return our results
      results)))

#_
(def example-polyline "pdymEssfy[rJhAlANNeALyBD{@XqFBW^[email protected]@[email protected]@[email protected]@[email protected]{@[email protected]@[email protected][email protected][email protected]@[email protected]`[email protected]@[email protected]@[[email protected]@[email protected]@][email protected]}DmBwD{[email protected]@}[email protected]@wDK}@@[[email protected]}[email protected]@[email protected]{[email protected]@[email protected]@[email protected]@{[email protected]{[email protected]^[email protected]}AdAcA|@[email protected]@[email protected]@Y|[email protected]@[email protected]@[email protected]@[email protected]}@[email protected]@eCVeDRiGF}FGiB[[email protected]{[email protected]@[email protected]@[email protected]@uByA}[email protected]{[email protected]@[email protected]@[email protected]@}[email protected]@[email protected][email protected][email protected]@kD^}A`@uA`CiHPcABYB[[email protected]@][email protected]@[email protected]@[email protected]@[email protected]][email protected][email protected]][email protected]@[email protected]@{@[email protected]@[email protected][email protected]@[email protected]@GK][email protected]@[email protected]}@[email protected]@[email protected][[email protected][UQ[[email protected]_DKsCCcBJcDcCIuBK")
#_
(path-for-encoded-polyline (seq (char-array example-polyline)))


Permalink - Comments - Tags: Development,Clojure

[First Page] [Prev] Showing page 1 of 1 pages [Next] [Last Page]

Tags

Archive

© 2008 Tom Horn