This page is not created by, affiliated with, or supported by Slack Technologies, Inc.
2022-03-17
Channels
- # announcements (6)
- # babashka (2)
- # babashka-sci-dev (1)
- # beginners (74)
- # calva (3)
- # cider (33)
- # clj-kondo (19)
- # cljsrn (10)
- # clojure (75)
- # clojure-dev (11)
- # clojure-europe (39)
- # clojure-italy (1)
- # clojure-nl (1)
- # clojure-spec (4)
- # clojure-uk (6)
- # clojurescript (139)
- # code-reviews (8)
- # core-typed (7)
- # data-science (1)
- # docs (2)
- # emacs (11)
- # events (1)
- # introduce-yourself (8)
- # lsp (4)
- # malli (10)
- # off-topic (15)
- # pedestal (5)
- # podcasts (4)
- # polylith (18)
- # re-frame (6)
- # react (1)
- # reagent (18)
- # reitit (6)
- # releases (2)
- # rewrite-clj (1)
- # spacemacs (15)
- # sql (2)
- # vscode (5)
I have an array of maps:
(def current-prices [{:item_no "A1", :unit_cost 0.00M}
{:item_no "A2", :unit_cost 0.00M}
{:item_no "A3", :unit_cost 938.51M}
{:item_no "A4", :unit_cost 213.38M}])
I would like to retrieve the unit_cost for a certain item_no, let’s say “A1”. How would I do that?You can filter the vector on the value of :item_no and then retrieve the :unit_cost value from the result
you might also consider thinking of this as a relation (set of collections) and using [clojure.set](https://clojuredocs.org/clojure.set)
If you're going to do this often, it may make sense to index them beforehand:
(defn index-by [f coll]
(persistent!
(reduce #(assoc! % (f %2) %2) (transient {}) coll)))
(def current-prices [{:item_no "A1", :unit_cost 0.00M}
{:item_no "A2", :unit_cost 0.00M}
{:item_no "A3", :unit_cost 938.51M}
{:item_no "A4", :unit_cost 213.38M}])
(def prices-by-id (index-by :item_no current-prices))
After this, gertting the price of one item given its :item_no
becomes a constant-time operation:
(:unit_cost (prices-by-id "A3"))
; => 938.51M
I usually reach for group-by
as the intermediate data transform for situations like this:
user=> (-> current-prices (->> (group-by :item_no)) (get "A1"))
[{:item_no "A1", :unit_cost 0.00M}]
also, if you plan to do this kind of search, maybe that data should be indexed by :item_no
in the first place
I’m trying to call the public method createScoped(Collection<String> scopes)
on the Java class GoogleCredentials
within Google’s library com.google.auth.oauth2
. I already have an instance of the class. This method expects a parameter of type Collection<String>
. How can I transform a clojure.lang.PersistentVector
to the Java type Collection<String>
?
I found out I can do this: (java.util.ArrayList. ["this is" "a vector" "of strings"])
Which returns a java.util.ArrayList
Not quite what I’m after, but maybe a step in the right direction…
Oh, I see… Looks like the Collection
and List
in Java are interfaces, implemented by classes such as ArrayList
, LinkedList
, Vector
, etc. Looks like I might have found the answer to my own question after all 😅 Now to try it out…
According to https://clojure.org/reference/data_structures:
> “The Java collection interfaces specify algorithms for Lists
, Sets
, and Maps
in calculating hashCode()
values. All Clojure collections conform to these specifications in their hashCode()
implementations.”
Interesting! Does that mean I can pass a Clojure vector straight into that Java class method? :thinking_face: That thought didn’t even occur to me…
Hahaha! Indeed. I could simply pass the Clojure vector straight into the Java class method, and that worked. Now I feel really dumb for not trying that sooner, and getting hung up on Google’s documentation 😅 Oh, well! Now I’ll never forget about that. Learning the hard way! Lesson of the day: Try more in the REPL right away; think less about documentation.
(def key (io/input-stream "key.json"))
(def credentials (GoogleCredentials/fromStream key))
(def scopes ["these are" "not" "valid scopes"])
(def scoped-credentials (. credentials createScoped scopes))
Hello everyone!
I have this map {"map1" {:a 1, :b 2}, "map2" {:c 3, :d 4}}
named "nested-map"
How do I assoc some key :e with value 5 inside the key of "map1", how can I do so?
I tried some like this (assoc (get nested-map "map1") :a 3)
but it returned {:a 3, :b 2}
and I want the whole map with the change to return
update-in
usually use in scenario that require to manipulate existing value, of course the same effect can be achieved as (update-in nested-map ["map1" :e] (constantly 5))
, but is a bit clumsy than above solution using assoc-in
, well just my two cents 🤞:skin-tone-2:
For sure! I meant as a general point, since OP didn’t know about assoc-in and update-in is quite a common and useful thing.
I’m calling several Java methods using the thread-first macro (`->`). One of those methods can either return an updated object or nil
. Is there a way to “skip” that method in the thread-first macro if it returns nil
, so that nil
does not get passed to the next function in the pipeline? I’m imagining something analogous to a “predicate guard” in Elixir pattern-matching. Because if nil
is passed to the next function in the pipeline, it will crash. I only want the updated object if there is one.
Awesome, thanks @U01GQRC8W30! I was not aware of that macro. It doesn’t seem to work the way I expected, however. Here’s my function (WIP):
(defn get-access-token [key-path scopes]
(let [k (io/input-stream key-path) s scopes]
(some-> k
(GoogleCredentials/fromStream)
(. createScoped s)
(. refreshIfExpired) ;; This is the offender!
(. getAccessToken))))
The function refreshIfExpired
could return nil
, which would short-circuit the pipeline if I use some->
. And then getAccessToken
would not get called.
Note that I need it to execute getAccessToken
no matter what. Just “skip” the one step refreshIfExpired
if it returns nil
.Perhaps a better solution would be to wrap refreshIfExpired
in my own function or create a separate function for handling the token refresh if it has expired. Then include my own function in the thread-first macro, instead of using refreshIfExpired
directly in there :thinking_face:
This is useful in many situations, but it would be nice to have some info from the point the nil was introduced
Like Either[L,R] in Scala, composing it gives info about the stage in the overall thing that produced a failure
What is the "Clojuronic" way to do this?
@U035WLPF552 If your question is for me, I’m afraid I’m not experienced enough with Scala or Clojure (yet) to give you an answer. But perhaps someone else might chime in. But… Back to the original problem I was trying to solve. I think I’ve found a nicer solution, along the lines of what I described above. Here is what I came up with:
(defn get-access-token [credentials]
(if-let [c (. credentials refreshIfExpired)]
(. c getAccessToken)
(. credentials refreshAccessToken)))
(defn get-credentials [json-key-path google-api-scopes]
(let [k (io/input-stream json-key-path) s google-api-scopes]
(-> k
(GoogleCredentials/fromStream)
(. createScoped s)
(get-access-token))))
But for some unknown reason, when I call my function get-credentials
, I get a new access token every time, even though the previous one has not expired (I have checked the time-to-live).
The if-let
in get-access-token is working correctly (I have verified this by stepping through the code with the debugger). Not sure what’s going on here…
For context, here is Google’s own Java example, which I’m using as a reference:
GoogleCredentials credentials = GoogleCredentials.fromStream(new FileInputStream("/path/to/credentials.json"));
credentials.refreshIfExpired();
AccessToken token = credentials.getAccessToken();
// OR
AccessToken token = credentials.refreshAccessToken();
Figured it out. Here’s the “final” version, where the token does not get refreshed needlessly every time the function is invoked:
(defn get-access-token [credentials]
(. credentials refreshIfExpired)
(. credentials getAccessToken))
(defn get-credentials [json-key-path google-api-scopes]
(let [k (io/input-stream json-key-path) s google-api-scopes]
(-> k
(ServiceAccountCredentials/fromStream)
(. createScoped s))))
(def my-credentials (get-credentials
"secrets/service_account_key.json"
[""]))
(def my-token (get-access-token my-credentials))
my-token
The observant reader will notice that I switched to ServiceAccountCredentials
, and I had misunderstood how refreshIfExpired
worked. Upon reading the documentation more closely, I discovered that this method never returns anything; it just updates the object’s cached token. So the if-let
I had written earlier was redundant.@U01PE7630AC since this is #beginners -- style tip, using .
is low level and usually only done in macros, instead of (. instance someMethod)
the normal thing is (.someMethod instance)
- the compiler translates it for you
I know it reads weird coming from java where instance.someMethod()
and (. instance someMethod)
look very similar, but it's more in line with normal clojure code, where the first thing in the list is always the operation being invoked
Aha, nice! Thanks for the advice, @U051SS2EU! Much appreciated.
In a way, it does make more sense to say “call this method in this object,” than “in this object, call this method.” And it makes sense that the first thing in the list is the “invokable thing” (function/macro/special form), to keep things consistent. I like it!
in truth that's the case both times, but that's why .
is lower level - it says "generate a method call against this object for that method", where you could simply say "invoke this method on that object" and let clojure translate it
By the way… Is there a simple way to browse/see which classes and methods are available from Java libraries, directly from within the code editor or REPL? (I’m using Visual Studio Code with Calva.)
I’m currently using the scattered and partially outdated/misleading Google web pages and GitHub README.md
files… It’s a bit cumbersome.
To figure out the correct include-paths, I’ve been using this command-line trick:
$ jar -tf ~/.m2/repository/com/google/auth/google-auth-library-oauth2-http/1.6.0/google-auth-library-oauth2-http-1.6.0.jar | grep ServiceAccountCredentials
com/google/auth/oauth2/ServiceAccountCredentials$Builder.class
com/google/auth/oauth2/ServiceAccountCredentials$1.class
com/google/auth/oauth2/ServiceAccountCredentials.class
Then copying the path (e.g., com/google/auth/oauth2
), changing the /
to .
to be included with by ns
in my Clojure source, like so:
(ns google-api-client
(:import [com.google.auth.oauth2 ServiceAccountCredentials]))
in vim or emacs (at least, not sure about other editors) you can open a jar file and see a listing with the class files
After spending 2 workdays, I’m giving up on my project of using Google My Business API via the https://developers.google.com/api-client-library/java in Clojure. Not because of Clojure, but because Google’s API is incredibly cumbersome to use. 2-3 new classes pop up for each step… All method parameters and return values are some obscure custom object that needs to be imported, like HttpTransport, JsonFactory, HttpRequestInitializer, MyBusinessAccountManagement$Builder… It never ends 😅 And to add insult to injury, all Google URLs require https://google.aip.dev/127. Think I need to find a more pleasant first project, sans complex Java libs.
I never use import for intermediate types, only the ones I need to construct / call static methods on / type hint for perf after measuring
@U051SS2EU Thanks again for the advice! Some methods in the Java library require special objects as input parameters. For example, the constructor for MyBusinessAccountManagement$Builder
requires an instance of GoogleNetHttpTransport
, GsonFactory
and HttpCredentialsAdapter
. And to instantiate HttpCredentialsAdapter
, I need an instance of ServiceAccountCredentials
, and so on. Is that what you mean by “intermediate types?” If there is a way to get around that, it would make things way easier!
Here is a comment block illustrating my current (and probably naive) approach:
(comment
(def http-transport (GoogleNetHttpTransport/newTrustedTransport))
(def json-factory (GsonFactory/getDefaultInstance))
(def credentials (get-credentials
"secrets/service_account_key.json"
[""]))
(def http-request-initializer (new HttpCredentialsAdapter credentials))
(def builder (new MyBusinessAccountManagement$Builder
http-transport
json-factory
http-request-initializer)))
And for comparison, here is the working Python solution which I implemented in <15 mins:
SERVICE_ACCOUNT_KEY = 'secrets/service_account_key.json'
SCOPES = ['']
API_SERVICE_NAME = 'mybusinessaccountmanagement'
API_VERSION = 'v1'
credentials = service_account.Credentials.from_service_account_file(
SERVICE_ACCOUNT_KEY, scopes=SCOPES)
service = build(API_SERVICE_NAME, API_VERSION, credentials=credentials)
response = service.accounts().list().execute()
OK - you aren't doing the thing I was concerned about (wouldn't have even mentioned if the channel wasn't #beginners) - the only thing I would change there is that I'd never put runtime data into def (but this is easier when doing a first draft and experimenting of course)
also new
is relateively low level, and instead of (new SomeObject)
the idiomatic thing is (SomeObject.)
this is one of the cases where the official clojure documentation is very good https://clojure.org/reference/java_interop
Awesome, thanks again for taking the time to let me know! Yeah, that very linear/imperative approach with putting runtime data in def
is just me trying to figure out how these classes and objects work by playing around in the REPL. My plan was to rewrite it and create some function “verbs” after figuring out the correct order of operations and required objects.
an alternate approach is to create the data in a lexical context (let block and/or function args) and then use tap>
to capture the values
eg.
(def registry (atom []))
(defn capture
[{:keys [description payload]}]
(swap! registry conj
{:desc description
:data payload
:at (System/currentTimeMillis)})
(add-tap #'capture) ;; using the var makes it easier to remove the tap
(defn do-something
[]
(let [foo (SomeObject.)
bar (OtherThing. foo)]
(tap> {:description "do something local context"
:payload {:foo foo
:bar bar}})
(frob bar))
if you don't add a tapping function, tap> is basically free, so you don't need to remove the tap call for production - just the call to add-tapgiven that we use immutable data so often, this provides at least 80% of what you'd need from a step debugger also
Thanks again for the advice! Those are some concepts I haven’t encountered yet, so I’ll need to spend some time digging into them.
Also… I noticed that Google provides various listings and “meta-APIs,” which can be used to get info about the APIs, i.e., which APIs are available, their inputs and outputs, etc. I feel like this is a case where an experienced Clojure programmer would just “magic” a “dynamic wrapper” for the entire Google API in 50 lines of Clojure code by using those. 😂
there are projects that generate wrappers using clojure.reflect/reflect
- but the magic comes at a cost, when you hit an error the stack trace won't lead you to any useful source code to look at
@U01PE7630AC also - here's an example of me using r/reflect - I'm showing the exploration I did as well as the final result, as an example of an exploratory repl workflow hope it's obvious that I'm not retyping each line, but just using up-arrow then editing, the repl terminal becomes a narrative describing my exploration, and the repl history makes it easy to recreate it
(cmd)user=> (require '[clojure.reflect :as r])
nil
(cmd)user=> (-> (java.util.Date.) (r/reflect) keys)
(:bases :flags :members)
(cmd)user=> (-> (java.util.Date.) (r/reflect) :members type)
clojure.lang.PersistentHashSet
(cmd)user=> (-> (java.util.Date.) (r/reflect) :members first)
#clojure.reflect.Method{:name getCalendarSystem, :return-type sun.util.calendar.BaseCalendar, :declaring-class java.util.Date, :parameter-types [int], :exception-types [], :flags #{:private :static :final}}
(cmd)user=> (-> (java.util.Date.) (r/reflect) :members (->> (map :name)))
(getCalendarSystem getTimeImpl setTime getCalendarSystem getDate clone before java.util.Date ttb getHours compareTo java.util.Date normalize java.util.Date setMonth java.util.Date java.util.Date wtb serialVersionUID setMinutes fastTime normalize java.util.Date defaultCenturyStart setYear toGMTString getMonth setHours toString getDay getCalendarDate compareTo toLocaleString getJulianCalendar parse getYear toInstant UTC setSeconds jcal getMillisOf equals readObject getTimezoneOffset gcal after convertToAbbr writeObject getMinutes setDate getTime getCalendarSystem from hashCode getSeconds cdate)
(cmd)user=> (-> (java.util.Date.) (r/reflect) :members (->> (group-by :name)) (get 'parse) pprint)
[{:name parse,
:return-type long,
:declaring-class java.util.Date,
:parameter-types [java.lang.String],
:exception-types [],
:flags #{:public :static}}]
nil
I used keys
because I remembered that it returned a hash-map but didn' t remember the key I want,, type
because I didn't remember the data type of the member entry collection, first
to get an example of what each member descriptor contains, map
to browse the method and public slot names, finally group-by
and pprint
to ensure I saw all the overloads of the interesting method, in a readable form
That’s an interesting way of “digging” into the guts of stuff to figure things out when necessary via the REPL!
it really does feel like digging, at each step the chain of functions inside ->
gets a bit longer as I peek deeper into the data
and there's no need to switch gears, the way I do when looking up and reading docs- I'm using the same data manipulation I would in normal code
I suppose it would be possible to write a function using reflection to automatically “unpack” a thing, find its constituent parts, then call itself on each of those parts to N depth. Like a “recursive auto-unpacker.” Basically emulating/automating your process shown above.
I think the problem is that you'd end up with a whole bunch of descriptions of the methods on java.lang.Integer and java.lang.String and byte-array
Right… I guess it would be difficult to filter out the “noise” or highlight the interesting bits.
right, the interaction is me zooming in on the parts I care about, and to me that's more managable
there's also bean
(mentioned in the interop link above) which is convenient though limited
user=> (bean (java.util.Date.))
{:day 5, :date 18, :time 1647631253057, :month 2, :seconds 53, :year 122, :class java.util.Date, :timezoneOffset 420, :hours 12, :minutes 20}
but great as a quick/handy view of what you can get out of an objectI wonder if it would be possible to use core.logic
in combination with reflection to “query” out the interesting parts. I’m not skilled enough with logic programming, but my naive intuition is telling me there might be some interesting ideas to explore there. Almost like a “reflection-based search engine” for “unpacking” Java libraries and visualizing their innards.
Here’s a relevant thread to the discussion we were having here last week: https://clojurians.slack.com/archives/CBE668G4R/p1648453567896769
I'm planning to begin developing a public presence showcasing both my open-source projects and some blog entries. I’m wondering where to start. For example, I could leverage GitHub pages for at least some time (eventually transitioning to AWS/GCP). I suppose I could then just begin leveraging jekyll - though I’m learning a new framework for this anyway, so I have no prior commitment to jekyll.
there are a number of static site generators in clojure. I think cryogen is a popular one
I think bootleg is a newer one
@ipmonger I recently moved from Wordpress to Cryogen + Cloudflare pages - read here: https://curiousprogrammer.net/posts/2022-01-12-moving-to-cryogen Code for my blog is here: https://github.com/curiousprogrammer-net/curiousprogrammer.blog/