Fork me on GitHub
#beginners
<
2022-03-17
>
zeddan09:03:00

I have an array of maps:

(def current-prices [{:item_no "A1", :unit_cost 0.00M}
                     {:item_no "A2", :unit_cost 0.00M}
                     {:item_no "A3", :unit_cost 938.51M}
                     {:item_no "A4", :unit_cost 213.38M}])
I would like to retrieve the unit_cost for a certain item_no, let’s say “A1”. How would I do that?

vanelsas09:03:45

You can filter the vector on the value of :item_no and then retrieve the :unit_cost value from the result

vanelsas09:03:13

(get (first (filter #(= (:item_no %) "A1") current-prices) :unit_cost)

vanelsas09:03:19

Something like that ought to do it

zeddan09:03:04

Perfect, thank you!

dgb2310:03:40

you might also consider thinking of this as a relation (set of collections) and using [clojure.set](https://clojuredocs.org/clojure.set)

dgb2310:03:05

depending on what else you do with it

zxspectrr10:03:32

(->> current-prices
     (filter #(= (:item_no %) "A1"))
     first
     :unit_cost))

jaihindhreddy11:03:36

If you're going to do this often, it may make sense to index them beforehand:

(defn index-by [f coll]
  (persistent!
    (reduce #(assoc! % (f %2) %2) (transient {}) coll)))

(def current-prices [{:item_no "A1", :unit_cost 0.00M}
                     {:item_no "A2", :unit_cost 0.00M}
                     {:item_no "A3", :unit_cost 938.51M}
                     {:item_no "A4", :unit_cost 213.38M}])

(def prices-by-id (index-by :item_no current-prices))
After this, gertting the price of one item given its :item_no becomes a constant-time operation:
(:unit_cost (prices-by-id "A3"))
; => 938.51M

noisesmith19:03:32

I usually reach for group-by as the intermediate data transform for situations like this:

user=> (-> current-prices (->> (group-by :item_no)) (get "A1"))
[{:item_no "A1", :unit_cost 0.00M}]

noisesmith19:03:05

also, if you plan to do this kind of search, maybe that data should be indexed by :item_no in the first place

leifericf10:03:27

I’m trying to call the public method createScoped(Collection<String> scopes) on the Java class GoogleCredentials within Google’s library com.google.auth.oauth2. I already have an instance of the class. This method expects a parameter of type Collection<String>. How can I transform a clojure.lang.PersistentVector to the Java type Collection<String>?

leifericf11:03:08

I found out I can do this: (java.util.ArrayList. ["this is" "a vector" "of strings"]) Which returns a java.util.ArrayList Not quite what I’m after, but maybe a step in the right direction…

leifericf12:03:32

Oh, I see… Looks like the Collection and List in Java are interfaces, implemented by classes such as ArrayList, LinkedList, Vector, etc. Looks like I might have found the answer to my own question after all 😅 Now to try it out…

leifericf12:03:51

According to https://clojure.org/reference/data_structures: > “The Java collection interfaces specify algorithms for Lists, Sets, and Maps in calculating hashCode() values. All Clojure collections conform to these specifications in their hashCode() implementations.” Interesting! Does that mean I can pass a Clojure vector straight into that Java class method? :thinking_face: That thought didn’t even occur to me…

leifericf12:03:17

Hahaha! Indeed. I could simply pass the Clojure vector straight into the Java class method, and that worked. Now I feel really dumb for not trying that sooner, and getting hung up on Google’s documentation 😅 Oh, well! Now I’ll never forget about that. Learning the hard way! Lesson of the day: Try more in the REPL right away; think less about documentation.

(def key (io/input-stream "key.json"))

(def credentials (GoogleCredentials/fromStream key))

(def scopes ["these are" "not" "valid scopes"])

(def scoped-credentials (. credentials createScoped scopes))

Adir Ohayon12:03:51

Hello everyone! I have this map {"map1" {:a 1, :b 2}, "map2" {:c 3, :d 4}} named "nested-map" How do I assoc some key :e with value 5 inside the key of "map1", how can I do so? I tried some like this (assoc (get nested-map "map1") :a 3) but it returned {:a 3, :b 2} and I want the whole map with the change to return

Godwin Ko12:03:27

(assoc-in nested-map ["map1" :e] 5)

Adir Ohayon12:03:42

Awesome! thank you for the fast reply 😄

1
dgb2314:03:53

also have a look at update-in

Godwin Ko14:03:17

update-in usually use in scenario that require to manipulate existing value, of course the same effect can be achieved as (update-in nested-map ["map1" :e] (constantly 5)), but is a bit clumsy than above solution using assoc-in, well just my two cents 🤞:skin-tone-2:

☝️ 1
dgb2317:03:26

For sure! I meant as a general point, since OP didn’t know about assoc-in and update-in is quite a common and useful thing.

1
leifericf15:03:17

I’m calling several Java methods using the thread-first macro (`->`). One of those methods can either return an updated object or nil. Is there a way to “skip” that method in the thread-first macro if it returns nil, so that nil does not get passed to the next function in the pipeline? I’m imagining something analogous to a “predicate guard” in Elixir pattern-matching. Because if nil is passed to the next function in the pipeline, it will crash. I only want the updated object if there is one.

leifericf15:03:51

Awesome, thanks @U01GQRC8W30! I was not aware of that macro. It doesn’t seem to work the way I expected, however. Here’s my function (WIP):

(defn get-access-token [key-path scopes]
  (let [k (io/input-stream key-path) s scopes]
    (some-> k
        (GoogleCredentials/fromStream)
        (. createScoped s)
        (. refreshIfExpired) ;; This is the offender!
        (. getAccessToken))))
The function refreshIfExpired could return nil, which would short-circuit the pipeline if I use some->. And then getAccessToken would not get called. Note that I need it to execute getAccessToken no matter what. Just “skip” the one step refreshIfExpired if it returns nil.

leifericf15:03:51

Perhaps a better solution would be to wrap refreshIfExpired in my own function or create a separate function for handling the token refresh if it has expired. Then include my own function in the thread-first macro, instead of using refreshIfExpired directly in there :thinking_face:

Danilo Oliveira15:03:17

This is useful in many situations, but it would be nice to have some info from the point the nil was introduced

Danilo Oliveira15:03:57

Like Either[L,R] in Scala, composing it gives info about the stage in the overall thing that produced a failure

Danilo Oliveira16:03:37

What is the "Clojuronic" way to do this?

leifericf16:03:22

@U035WLPF552 If your question is for me, I’m afraid I’m not experienced enough with Scala or Clojure (yet) to give you an answer. But perhaps someone else might chime in. But… Back to the original problem I was trying to solve. I think I’ve found a nicer solution, along the lines of what I described above. Here is what I came up with:

(defn get-access-token [credentials]
  (if-let [c (. credentials refreshIfExpired)]
    (. c getAccessToken)
    (. credentials refreshAccessToken)))

(defn get-credentials [json-key-path google-api-scopes]
  (let [k (io/input-stream json-key-path) s google-api-scopes]
    (-> k
        (GoogleCredentials/fromStream)
        (. createScoped s)
        (get-access-token))))
But for some unknown reason, when I call my function get-credentials, I get a new access token every time, even though the previous one has not expired (I have checked the time-to-live). The if-let in get-access-token is working correctly (I have verified this by stepping through the code with the debugger). Not sure what’s going on here… For context, here is Google’s own Java example, which I’m using as a reference:
GoogleCredentials credentials = GoogleCredentials.fromStream(new FileInputStream("/path/to/credentials.json"));
credentials.refreshIfExpired();
AccessToken token = credentials.getAccessToken();
// OR
AccessToken token = credentials.refreshAccessToken();

leifericf17:03:17

Figured it out. Here’s the “final” version, where the token does not get refreshed needlessly every time the function is invoked:

(defn get-access-token [credentials]
  (. credentials refreshIfExpired)
  (. credentials getAccessToken))

(defn get-credentials [json-key-path google-api-scopes]
  (let [k (io/input-stream json-key-path) s google-api-scopes]
    (-> k
        (ServiceAccountCredentials/fromStream)
        (. createScoped s))))

(def my-credentials (get-credentials
                     "secrets/service_account_key.json"
                     [""]))

(def my-token (get-access-token my-credentials))

my-token
The observant reader will notice that I switched to ServiceAccountCredentials, and I had misunderstood how refreshIfExpired worked. Upon reading the documentation more closely, I discovered that this method never returns anything; it just updates the object’s cached token. So the if-let I had written earlier was redundant.

noisesmith19:03:42

@U01PE7630AC since this is #beginners -- style tip, using . is low level and usually only done in macros, instead of (. instance someMethod) the normal thing is (.someMethod instance) - the compiler translates it for you

noisesmith19:03:48

I know it reads weird coming from java where instance.someMethod() and (. instance someMethod) look very similar, but it's more in line with normal clojure code, where the first thing in the list is always the operation being invoked

leifericf19:03:37

Aha, nice! Thanks for the advice, @U051SS2EU! Much appreciated.

leifericf19:03:03

In a way, it does make more sense to say “call this method in this object,” than “in this object, call this method.” And it makes sense that the first thing in the list is the “invokable thing” (function/macro/special form), to keep things consistent. I like it!

noisesmith19:03:43

in truth that's the case both times, but that's why . is lower level - it says "generate a method call against this object for that method", where you could simply say "invoke this method on that object" and let clojure translate it

👍 1
leifericf19:03:59

By the way… Is there a simple way to browse/see which classes and methods are available from Java libraries, directly from within the code editor or REPL? (I’m using Visual Studio Code with Calva.) I’m currently using the scattered and partially outdated/misleading Google web pages and GitHub README.md files… It’s a bit cumbersome. To figure out the correct include-paths, I’ve been using this command-line trick:

$ jar -tf ~/.m2/repository/com/google/auth/google-auth-library-oauth2-http/1.6.0/google-auth-library-oauth2-http-1.6.0.jar | grep ServiceAccountCredentials
com/google/auth/oauth2/ServiceAccountCredentials$Builder.class
com/google/auth/oauth2/ServiceAccountCredentials$1.class
com/google/auth/oauth2/ServiceAccountCredentials.class
Then copying the path (e.g., com/google/auth/oauth2), changing the / to . to be included with by ns in my Clojure source, like so:
(ns google-api-client
  (:import [com.google.auth.oauth2 ServiceAccountCredentials]))

noisesmith20:03:52

in vim or emacs (at least, not sure about other editors) you can open a jar file and see a listing with the class files

👍 1
noisesmith20:03:33

there's also the javadoc function which works quite often

👀 1
leifericf22:03:36

After spending 2 workdays, I’m giving up on my project of using Google My Business API via the https://developers.google.com/api-client-library/java in Clojure. Not because of Clojure, but because Google’s API is incredibly cumbersome to use. 2-3 new classes pop up for each step… All method parameters and return values are some obscure custom object that needs to be imported, like HttpTransport, JsonFactory, HttpRequestInitializer, MyBusinessAccountManagement$Builder… It never ends 😅 And to add insult to injury, all Google URLs require https://google.aip.dev/127. Think I need to find a more pleasant first project, sans complex Java libs.

noisesmith18:03:23

I never use import for intermediate types, only the ones I need to construct / call static methods on / type hint for perf after measuring

1
noisesmith18:03:54

mentioning this in case you are importing the intermediate types

👍 1
leifericf18:03:36

@U051SS2EU Thanks again for the advice! Some methods in the Java library require special objects as input parameters. For example, the constructor for MyBusinessAccountManagement$Builder requires an instance of GoogleNetHttpTransport, GsonFactory and HttpCredentialsAdapter. And to instantiate HttpCredentialsAdapter, I need an instance of ServiceAccountCredentials, and so on. Is that what you mean by “intermediate types?” If there is a way to get around that, it would make things way easier! Here is a comment block illustrating my current (and probably naive) approach:

(comment
  (def http-transport (GoogleNetHttpTransport/newTrustedTransport))

  (def json-factory (GsonFactory/getDefaultInstance))

  (def credentials (get-credentials
                    "secrets/service_account_key.json"
                    [""]))
  
  (def http-request-initializer (new HttpCredentialsAdapter credentials))

  (def builder (new MyBusinessAccountManagement$Builder
                    http-transport
                    json-factory
                    http-request-initializer)))
And for comparison, here is the working Python solution which I implemented in <15 mins:
SERVICE_ACCOUNT_KEY = 'secrets/service_account_key.json'
SCOPES = ['']
API_SERVICE_NAME = 'mybusinessaccountmanagement'
API_VERSION = 'v1'

credentials = service_account.Credentials.from_service_account_file(
    SERVICE_ACCOUNT_KEY, scopes=SCOPES)

service = build(API_SERVICE_NAME, API_VERSION, credentials=credentials)

response = service.accounts().list().execute()

noisesmith18:03:20

OK - you aren't doing the thing I was concerned about (wouldn't have even mentioned if the channel wasn't #beginners) - the only thing I would change there is that I'd never put runtime data into def (but this is easier when doing a first draft and experimenting of course)

👍 1
noisesmith18:03:00

also new is relateively low level, and instead of (new SomeObject) the idiomatic thing is (SomeObject.)

🙏 1
noisesmith18:03:59

this is one of the cases where the official clojure documentation is very good https://clojure.org/reference/java_interop

❤️ 1
leifericf18:03:57

Awesome, thanks again for taking the time to let me know! Yeah, that very linear/imperative approach with putting runtime data in def is just me trying to figure out how these classes and objects work by playing around in the REPL. My plan was to rewrite it and create some function “verbs” after figuring out the correct order of operations and required objects.

noisesmith18:03:17

an alternate approach is to create the data in a lexical context (let block and/or function args) and then use tap> to capture the values eg.

(def registry (atom []))

(defn capture
  [{:keys [description payload]}]
  (swap! registry conj
     {:desc description
      :data payload
      :at (System/currentTimeMillis)})

(add-tap #'capture) ;; using the var makes it easier to remove the tap

(defn do-something
   []
   (let [foo (SomeObject.)
         bar (OtherThing. foo)]
     (tap> {:description "do something local context"
            :payload {:foo foo
                      :bar bar}})
     (frob bar))
if you don't add a tapping function, tap> is basically free, so you don't need to remove the tap call for production - just the call to add-tap

noisesmith18:03:34

given that we use immutable data so often, this provides at least 80% of what you'd need from a step debugger also

leifericf18:03:53

Thanks again for the advice! Those are some concepts I haven’t encountered yet, so I’ll need to spend some time digging into them.

leifericf18:03:02

Also… I noticed that Google provides various listings and “meta-APIs,” which can be used to get info about the APIs, i.e., which APIs are available, their inputs and outputs, etc. I feel like this is a case where an experienced Clojure programmer would just “magic” a “dynamic wrapper” for the entire Google API in 50 lines of Clojure code by using those. 😂

noisesmith19:03:29

there are projects that generate wrappers using clojure.reflect/reflect - but the magic comes at a cost, when you hit an error the stack trace won't lead you to any useful source code to look at

😮 1
noisesmith19:03:55

@U01PE7630AC also - here's an example of me using r/reflect - I'm showing the exploration I did as well as the final result, as an example of an exploratory repl workflow hope it's obvious that I'm not retyping each line, but just using up-arrow then editing, the repl terminal becomes a narrative describing my exploration, and the repl history makes it easy to recreate it

(cmd)user=> (require '[clojure.reflect :as r])
nil
(cmd)user=> (-> (java.util.Date.) (r/reflect) keys)
(:bases :flags :members)
(cmd)user=> (-> (java.util.Date.) (r/reflect) :members type)
clojure.lang.PersistentHashSet
(cmd)user=> (-> (java.util.Date.) (r/reflect) :members first)
#clojure.reflect.Method{:name getCalendarSystem, :return-type sun.util.calendar.BaseCalendar, :declaring-class java.util.Date, :parameter-types [int], :exception-types [], :flags #{:private :static :final}}
(cmd)user=> (-> (java.util.Date.) (r/reflect) :members (->> (map :name)))
(getCalendarSystem getTimeImpl setTime getCalendarSystem getDate clone before java.util.Date ttb getHours compareTo java.util.Date normalize java.util.Date setMonth java.util.Date java.util.Date wtb serialVersionUID setMinutes fastTime normalize java.util.Date defaultCenturyStart setYear toGMTString getMonth setHours toString getDay getCalendarDate compareTo toLocaleString getJulianCalendar parse getYear toInstant UTC setSeconds jcal getMillisOf equals readObject getTimezoneOffset gcal after convertToAbbr writeObject getMinutes setDate getTime getCalendarSystem from hashCode getSeconds cdate)
(cmd)user=> (-> (java.util.Date.) (r/reflect) :members (->> (group-by :name)) (get 'parse) pprint)
[{:name parse,
  :return-type long,
  :declaring-class java.util.Date,
  :parameter-types [java.lang.String],
  :exception-types [],
  :flags #{:public :static}}]
nil

😮 1
noisesmith19:03:08

I used keys because I remembered that it returned a hash-map but didn' t remember the key I want,, type because I didn't remember the data type of the member entry collection, first to get an example of what each member descriptor contains, map to browse the method and public slot names, finally group-by and pprint to ensure I saw all the overloads of the interesting method, in a readable form

👍 1
leifericf19:03:10

That’s an interesting way of “digging” into the guts of stuff to figure things out when necessary via the REPL!

noisesmith19:03:56

it really does feel like digging, at each step the chain of functions inside -> gets a bit longer as I peek deeper into the data

👍 1
noisesmith19:03:42

and there's no need to switch gears, the way I do when looking up and reading docs- I'm using the same data manipulation I would in normal code

leifericf19:03:12

I suppose it would be possible to write a function using reflection to automatically “unpack” a thing, find its constituent parts, then call itself on each of those parts to N depth. Like a “recursive auto-unpacker.” Basically emulating/automating your process shown above.

noisesmith19:03:37

I think the problem is that you'd end up with a whole bunch of descriptions of the methods on java.lang.Integer and java.lang.String and byte-array

👍 1
leifericf19:03:12

Right… I guess it would be difficult to filter out the “noise” or highlight the interesting bits.

noisesmith19:03:38

right, the interaction is me zooming in on the parts I care about, and to me that's more managable

👍 1
noisesmith19:03:30

there's also bean (mentioned in the interop link above) which is convenient though limited

user=> (bean (java.util.Date.))
{:day 5, :date 18, :time 1647631253057, :month 2, :seconds 53, :year 122, :class java.util.Date, :timezoneOffset 420, :hours 12, :minutes 20}
but great as a quick/handy view of what you can get out of an object

😮 1
leifericf19:03:07

I wonder if it would be possible to use core.logic in combination with reflection to “query” out the interesting parts. I’m not skilled enough with logic programming, but my naive intuition is telling me there might be some interesting ideas to explore there. Almost like a “reflection-based search engine” for “unpacking” Java libraries and visualizing their innards.

noisesmith19:03:52

OMG I just had the horrifying though of instance method SEO

😂 1
leifericf19:03:55

So many rabbit holes 😂

leifericf16:03:50

Here’s a relevant thread to the discussion we were having here last week: https://clojurians.slack.com/archives/CBE668G4R/p1648453567896769

Jon Boone20:03:41

I'm planning to begin developing a public presence showcasing both my open-source projects and some blog entries. I’m wondering where to start. For example, I could leverage GitHub pages for at least some time (eventually transitioning to AWS/GCP). I suppose I could then just begin leveraging jekyll - though I’m learning a new framework for this anyway, so I have no prior commitment to jekyll.

Cora (she/her)21:03:35

there are a number of static site generators in clojure. I think cryogen is a popular one

Cora (she/her)21:03:25

I think bootleg is a newer one

jumar22:03:48

@ipmonger I recently moved from Wordpress to Cryogen + Cloudflare pages - read here: https://curiousprogrammer.net/posts/2022-01-12-moving-to-cryogen Code for my blog is here: https://github.com/curiousprogrammer-net/curiousprogrammer.blog/

dgb2322:03:09

The pull request screenshot breaks your layout on mobile. You might want to set a max width on it

jumar05:03:05

Looks fine on my phone I guess. What does it look like for you?

dgb2310:03:32

I get a horizontal scroll bar

dgb2310:03:05

Iphone xs/webkit/firefox