I am saving a map into a JSONB postgres column but was having issues with cheshire so decided to use clojure.data.json since the custom encoder/decoder seemed a bit more straight forward. What is wrong with this solution?

(def opt_data {:sheet-data-structure
               {"RD" {[1 4] {:backgroundColor "#0000"}
                      :skip #{nil [5 5] [6 6]}}
                "Model" {:mods {[3 4] {:borderRight "none"}}
                         :test [1 2 3]}
                "Tax" {[99 9] ":should-not-be-keyword"}}})

(defn value-serializer
  [_ v]
  (cond
    (= (type v)
       (type #{})) (str "#SET~" v)
    :else          v))

(defn value-deserializer
  [_ v]
  (cond
    (str/starts-with? v "#SET~") (-> (subs v 5) read-string set)
    :else v))

(defn keys-serializer
  [k]
  (cond
    (keyword? k) (str "#KEYWORD~" k)
    (vector? k)  (str "#VECTOR~" k)
    :else        k))

(defn keys-deserializer
  [k]
  (cond
    (str/starts-with? k "#KEYWORD~") (-> k (subs 10) keyword)
    (str/starts-with? k "#VECTOR~") (-> k (subs 8read-string)
    :else k))

(-> (json/write-str opt_data :value-fn value-serializer :key-fn keys-serializer)
    (json/read-str :value-fn value-deserializer :key-fn keys-deserializer))

The reason I am doing this is that I need to keep sets as sets and vector keys as vectors and not strings. This data I am reading from the DB is being merged in with other another data of the same structure.

(-> k (subs 8read-string)

and replaced with 8

Yea I kept looking back n forth and I think it was a slack thing

I'd use pr-str instead of str inside the serializer I'd use pr-str on the k in the serializer, because it can preserve things that str might not

(cmd)user=> (str (map inc [1 2 3]))
"clojure.lang.LazySeq@7c42"
(cmd)user=> (pr-str (map inc [1 2 3]))
"(2 3 4)"

Oh, good catch!

it's a common gotcha - a good rule of thumb is that if what you want is the string you'd see in the repl for a given object, you want pr-str and not str

Is this a bad approach though? Is there a better way of doing this that I am not seeing?

There is discussion tomorrow about whether to do this manually or using Transit. Personally I would rather use Transit since it works out of the box and can probably do the job faster

what transit creates when you ask for "json" is not as likely to be useful in a jsonb field (eg. its shorthand for backreference of repeated keys in one document)

but it definitely works and its tagged readers/writers are mature and tested unlike yours

yea thats main pushback

65ms vs 890ms lol

For writing

woops more like 65ms vs 250ms. I was counting the read and write

I got stuck with a problem. Could someone help? I would like to call a -main function from a clojure script file with a namespace, but I do not know to do that. Here is what I tried. I have a Code.clj with Code namespace and a -main function. My current dir is the same directory as the source code (`/tmp/test`)

$ pwd
/tmp/test
$ cat Code.clj 
(ns Code
  (:gen-class))

(defn -main []
  (println "hello world"))
$ clojure -m Code Code.clj 
Execution error (FileNotFoundException) at clojure.main/main (main.java:40).
Could not locate Code__init.class, Code.clj or Code.cljc on classpath.

Full report at:
/tmp/clojure-90572729667538797.edn
$ 

@thoneyvazul you need a deps.edn file adding the current directory to the classpass (you also need to add & args to your -main function args)

I'd be interesting on knowing if there's a way to modify classpath without a deps.edn file

Awesome, it works! Thank you @dromar56

How can I sum up all pair freq (bi-grams) to a map using reduce? (def pairs '(("ab" "bd") ("ab" "bc") ("ab" "bk"))) (def freq '(1 2 3)) (defn f [p f]) => {"ab" 6, "bd" 1, "bc" 2, "bk" 3} {:low 5 :lowest 2} pairs will be (('lo', 'ow'...) ('lo', 'st'))....etc freq (5, 2) applying f will yield: => {"lo" 7, "ow" 7 "we" 2,...etc}

vals will return a sequence of all the values in your map, with no keys.

Just use reduce on that.

I do not have a map

each pair is corresponding to it freq, so 'ab':1 then 'ab':2 then 'ab':3, total freq of 6

That's what freq returns.

A map of all the frequencies.

Unless I'm completely confused.

let's thread on this

is there a way in spec to add additional description?? I mean if i create (s/def application-id int?) i could add a written description and explain what is the application-id. Then the same “description” would also be shown in swagger

No, this is the highest voted item in the tracker though

So we are aware of the desire :)

with spec-tools, you can say:

(require '[spec-tools.core :as st])
(require '[spec-tools.swagger :as swagger])

(swagger/transform
  (st/spec
    {:spec integer?
     :name "integer"
     :description "it's an int"
     :swagger/default 42}))
;{:type "integer"
; :title "integer"
; :description "it's an int"
; :default 42}

okay, maybe I need to go and vote it also (if possible).. And thanks, I will check spec-tools out :thumbsup:

curious - are there any production-teams using Property Based Testing? if so - with/without TDD? mixing? what are your thoughts? new to it and thinking on adding to our pipeline..

@aviv what am Property Based Test?

is that any way to adjust max-body size in jetty served apps?

We have mostly a traditional test suite with a few generative/property based tests. We don't have a separate generative test suite that takes a long time to run, which I have seen mentioned as a best practice. Our generative tests if they potentially take forever to run default to a small iteration count, but can be increased using a environment variable if you want a more complete test run. I am definitely not doing TDD

using generatorsfrom test.check I'm receiving exception with error Couldn't satisfy such-that predicate after 100 tries. when I try to generate none-empty vectors with 'tokens' -> strings which are lowercased + not empty

(gen/such-that (complement empty?)
                                (gen/vector (gen/such-that lowercase? gen/string-alphanumeric 100)))

it is a limitation of such-that

such-that can only filter (remove stuff that doesn't match) from the given generator (string-alphanumeric in this case)

and pretty much every random string is going to be filtered out

it is much more efficient to generate exactly what you want, but that is usually going to require writing a custom generator

you could also transform the output of the generator instead of filtering, e.g. instead of throwing out all the generated strings that are not all lowercase, you can take whatever generated string and make it lower case

On the same topic, can you spec a key which would be a value? I used fn? as predicate, but who can I fake the return values, or define the return values?

I guess I could use a generator of a set.

use ifn? not fn?, it's much more general

what do you mean by "key which would be a value"?

{:f (fn [x] (* 2 x))}

How would you spec :f?

a function would be ifn? as the predicate

I still didn't understand "key which would be a value" -- did you mean "key whose value is a function?"

not sure if there was a word missing

@ghadi Yes, this is what I desired to mean

If I was just doing a API/ server in Clojure , are people still using ring/compojure? Any new templates people would recommend?

At work, our (REST-ish) API Server is Ring/Compojure (and Component and a bunch of other stuff). @thurmondmb

Thanks @seancorfield wasn't sure if there was anything new in the past year or so

If we were starting over, maybe we'd look at other libraries more focused on APIs. Maybe compojure-api, or reitit, or something. But Ring/Compojure is fine as basis for most stuff. And I almost never use a template when starting a project (even tho' I created boot-new and clj-new for that purpose!).

(well, I might use clj-new to create a very basic app template, but that's it)

FWIW, I like to start from the basics because most templates that do more than very simple stuff are far too opinionated for my tastes 🙂

If I went the ring/ compojure route I could then add in Clojure spec or schema for validation?

And a swagger docs dependency if I needed it later on?

Sure. It's all just libraries.

Even when I start from just a basic app template, I usually end up with dozens of dependencies by the time I'm finished. I just find it easier to build things up as I go.

Cool thanks

Also, clojure.spec is pulled in automatically if you're using Clojure 1.9 or 1.10 (1.10.1 is the current stable release, 1.10.2 is in alpha right now).

(I just checked our API project at work and it looks like it has 16 top-level dependencies that bring in around 150 total libraries via transitive dependencies -- most of those top-level deps are actually other subprojects at work, so second-level deps would be a more realistic measure which is around 90 libraries)

Thanks !!

(our "core" business project, reused by everything else, has almost 60 top-level dependencies)

I'm going to try to build it from the ground up. When I was working in Clojure full time , my boss created our server that way was well. This was also before a lot of templates were available