ok, so this is the full iceberg: I want to store a json recieved from the web in a nosql store, but I would like to store only the fields Im interested in. So I was planning to use jackson.

(defrecord Foo ^{JsonCreator true} [^{JsonProperty "field1"} field1])
(-> (ObjectMapper.)
    (.readValue json Foo))

Why make a record?

use something like Cheshire or data.json to read in as data @paulocuneo

ok, that will solve the serialization issue, but how would you remove the unwanted field/nested-fields?

data.json takes a json string and gives back a hash map, then you can dissoc and encode again

I was thinking of select-keys actually, writing something like a parser But since the json is kinda big it would have been the same to describe the full structure.

(defn clear-fields[{:keys[field1 field2 ...manyfields]}]
   {:field1 (clear-sub-field1 field1)
    :field2 (clear-sub-field2 field2)
   ...
})

how deep of a tree is it?

clojure.walk ?

maybe 4 levels deep

Is the structure known and static?

yes, it's known and static

I just want to avoid storing garbage into nosql

So it's just an optimization?

it's a security issue too

but yes

Would it be easier to whitelist things you want to keep or blacklist things you know you don't want?

If you use a whitelist you can reduce over the collection of paths and accumulate into a map

it would be like select keys but with deeply nested paths.

If the keys are predictable and are bad at all levels of nesting, it's easy to sanitize with clojure.walk

user=> (clojure.walk/postwalk (fn [x] (if (map? x) (dissoc x :a :c :e) x)) {:a {:b :f} :b {:c :g :d :l} :d {:d {:d :e}}})
{:b {:d :l}, :d {:d {:d :e}}}

on the other hand if the keys happen at predictable levels, update-in with dissoc might be more straightforward

I am trying out clj-async-profiler on a project, OS X with Oracle/Apple JDK 8, starting it with clojure command. I am trying with the option -Djol.tryWithSudo=true mentioned in clj-async-profiler's README, which causes a password prompt to appear in the terminal window where I started clojure when clj-async-profiler triggers some call that requires superuser privileges. I try to enter my password, but it seems that the clojure REPL is fighting the password prompt for the input characters, and clojure wins. It interprets my password as an unknown Clojure symbol. Any advice for improving on this?

how do I access instance fields set in constructor of a deftype?

e.g. in CLJS I can do:

(deftype Foo [bar])

(def foo (->Foo "baz"))

(.-bar foo) ;; => "baz"

@lilactown That works in clj too so I'm not sure what your question is?

I think the issue is that the field I want to access is marked as volatile-mutable

I can’t find much docs but stackoverflow leads me to believe it’s private

It is

Any mutable fields will be made private

To access, your deftype needs to extend interfaces or protocols that can expose the data (safely)

Mutable stuff is inherently unsafe so is not public by default

I find odd that lein cloverage is assigning 100% of test coverage to a file full of specs. But I have literally 0 tests in my project. How is that?

specs are all macros. maybe just compiling them is "coverage" ?

just a guess

I just did some first (naive) tests comparing fressian and transit. It seems Transit is more compact than Fressian. Does this sound correct? I had expected the opposite https://gist.github.com/jeroenvandijk/b963ef7438b1511ed4362cca022f68f6

doesn't transit cache keys in maps ? could be it. I am quite sure fressian doesn't. On the otherhand fressian allows to define your own handlers for custom types, so I imagine you can optimize things further.

yeah i guess the extensibility is the prime benefit then (maybe cpu performance, but i don't care too much)

you could also add snappy/lz4 or someting on top of fressian

fressian does some optimisations (varints & co), but nothing substantial

as far as I remember

i should experiment more and see what i can get out of it. for example I havent seen this feature working yet https://github.com/Datomic/fressian/wiki/Caching

i hoped a repeated object would be cached but that doesn't seem to be the case

https://github.com/clojure/data.fressian/wiki/Creating-custom-handlers

you can end up with quite compact representation of "types" you can control

ex not repeating keys in records and whatnot, but that's not free (as in you need to specify what/how), same goes for caching

I quite like transit nowadays, was super sceptical at first

I was thinking of making my own dictionary lookup for repeated items (for arbitrary objects) and have some kind of reference that needs to be replaced on a read. I have this working with transit, which is already quite compact. But Transit doesn't allow custom objects (forcing me to have hacky things like strings with underscore prefixes), so i was hoping Fressian could do this, but the basic respresentation is much bigger

Maybe i need to try harder 🙂

Transit is super good, but i have a special use case for a very compact representation

if you can trade some ressources for storage size, just compress the content 😛

quite a waste, but I don't know your constraints

hmm yeah actually if it runs on the jvm it can work. Maybe i should try traditional compression techniques instead indeed

ok one mystery for me though, why is the bytesize of (data->transit-bytes [1]) smaller than (data->transit-bytes 1) (i'm using the code in the gist above). It's 2 vs 6 bytes

if I have a symbol/string representing a java method, is it possible to call that method on an object in a macro without using read-string or reflection?

This is what I've attempted (assumes a constructor with no parameters, and setpairs returns a sequence of vectors of [<setter-symbol> <value to set>]

(defmacro m->obj [m class]
  (let [setters (set-pairs m class)]
    `(let [obj# (new ~class)]
       (doseq [[field# v#] ~setters]
         (doto obj# ((memfn field#) v#))
         ;;(doto obj# (. field# v#))
         )
       obj#)))

@jjttjj I've made something similar in cljfx: https://github.com/cljfx/cljfx/blob/master/src/cljfx/composite.clj#L98-L106

it avoids reflection by type-hinting symbol holding an instance: (with-meta (gensym "instance") {:tag type-expr})

ah! thanks, I think that'll be perfect

type-expr would just basically be the class-name or int for primatives right?. Nevermind, got it!

Build tooling power in 2019: deps.edn vs boot? I stopped following the evolution of Clojure build tooling in 2017. At the time, boot was the best choice from a standpoint of power / configurability, and I was quite happy switching to it over lein. Since then, I see that deps.edn and the clojure command line tool are being pushed as the new path forward. I found https://corfield.org/blog/2018/04/18/all-the-paths/ as a summation of the current "state of the world" from Apr 2018. While trying to understand how tools.deps is used to support dev flow for a backend microservice, I found https://github.com/clojure/tools.deps.alpha/wiki/Tools From observing the current state of tools.deps libraries, it seems that the recommendation for tools.deps.alpha is to use a Makefile and/or aliases to setup CLI executable "tasks" for your project (dev, uberjar, etc.) It seems that tools.deps doesn't provide a nice "Clojure build layer" similar to Boot, but leaves that up to other tools which may eventually wrap it. Unfortunately, those tools don't seem to exist, at least not at the power level of Boot. The best I can find is https://github.com/mbuczko/revolt My conclusion -- Boot is still going to be a lot easier to work with for a nice hot reloading clojure backend service. Many of boot's benefits seems achievable with tools.deps, but the path forward is not apparent and will require a lot of "roll your own" approach. Beyond the improvements in classpath handling, the benefits offered by tools.deps don't seem to get you significantly further than Boot.

tl;dr: tools.deps is still evolving, use Boot if you want to get off the ground (relatively) fast with a good foundation for a microservice ^ My question to this community -- is this conclusion right, or is there a part of the picture I am missing here?

I think that’s pretty right when you scope it to “hot reloading Clojure backend service”

But I think there are a large number of projects that are satisfied without using Boot or lein at all

That's fair. I notice tools.deps has made huge leaps WRT raw CLI apps & graalVM

And I'm looking forward to using pure tools.deps someday, but my hunch now is that I'd get more heartache than help

If you need it, feel free to ask in #tools-deps

you will have to make the leap at some point though, there's no stopping this train

^ I also get that impression, and I do agree with the vision behind tools.deps 🙂

i see many projects use a dual approach, use lein and tools.deps together

i'm not familiar with boot though

I use tools.deps and maven together too (which is pretty easy with -Spom)

Although that’s mostly in contrib projects which have a long maven history

Haha, maven gets much further out of my realm of know-how. I never got the "proper Java background" much of the community comes from. I do notice https://github.com/RickMoynihan/lein-tools-deps & https://github.com/seancorfield/boot-tools-deps as integrated approaches, though I'm not sure what I'd use 'em for in practice

I see this as a safe transition path, use tools.deps for its own power and services, use lein/boot for what you need that doesn't exist yet in this ecosystem

like your hot reloading

Makes sense. I'm interested in picking up https://juxt.pro/edge/index.html just to get off the ground, but I just noticed... they already made the leap from boot to deps.edn Even found https://github.com/juxt/edge/blob/master/examples/tutorial.vent/deps.edn as a SPA example with figwheel reloading.

I can probably run with this example & eventually pull in revolt for a Clojure build layer. Out of respect for the support & work going into tools.deps, I'll give this a shot and see if I can make it work. Thanks @carkh & @alexmiller for the input!

good luck !

@quest Happy to answer questions about workflow/tooling -- we started with lein at work back in 2011, switched completely to boot in late 2015, and switched completely to clj/`deps.edn` last year.

I've also switched most of my OSS projects over to deps.edn now (instead of lein or boot).

Still rocking that good old lein... what made you change @seancorfield?

I need to use a macro to call getter methods with no arguments on an object. The following works when passed a string but cannot be passed something that needs to be evaluated.

(defmacro getter [nm obj]
  (let [getter-expr (symbol (str "." nm))]
    `(~getter-expr ~obj)))

(getter "getMyValue" myObject) ;;works

(let [s "getMyValue"] (getter s  myObject)) ;;doesn't work

it can't be done without eval, as s is just the symbol s until runtime

with eval, you'll probably want a lambda that receives s as an argument, as eval doesn't see locals

gotcha, thank you!