This page is not created by, affiliated with, or supported by Slack Technologies, Inc.
2021-02-11
Channels
- # announcements (6)
- # babashka (61)
- # beginners (85)
- # calva (21)
- # cider (6)
- # clara (9)
- # clj-http (1)
- # clj-kondo (35)
- # cljfx (6)
- # clojure (91)
- # clojure-australia (11)
- # clojure-europe (23)
- # clojure-italy (7)
- # clojure-losangeles (2)
- # clojure-nl (27)
- # clojure-uk (107)
- # clojurescript (4)
- # community-development (1)
- # cursive (69)
- # emacs (12)
- # fulcro (29)
- # graalvm (25)
- # honeysql (10)
- # hugsql (3)
- # integrant (13)
- # jobs (4)
- # kaocha (3)
- # keechma (1)
- # lambdaisland (3)
- # leiningen (2)
- # meander (17)
- # mount (3)
- # observability (1)
- # off-topic (86)
- # pathom (3)
- # polylith (2)
- # practicalli (14)
- # reitit (14)
- # shadow-cljs (61)
- # startup-in-a-month (1)
- # tools-deps (9)
- # vim (54)
- # xtdb (16)
I have the following question posted in clj-http: https://clojurians.slack.com/archives/C8860D6BS/p1613018229002500 Your help is much appreciated!
@dpsutton what do you mean by real data? It’s key value pairs where values are strings
He means that :basic-auth
string I think...
That auth seems real (although the customer ID seems invalid)
You might want to delete that message @ps although at this point it's probably been archived to the logging site -- not sure if it respects deletes. The Zulip bot does respect deletes so it will at least remove it from there.
The #clj-http channel at least is not logged/archived as far as I can tell so at least it's only in this channel -- and Zulip doesn't copy the body of shared messages. Not sure if logbot does.
@ps You could try :body "customer= cus_IvNWjfw1q2clxS"
in the post request. It’s equivalent to --data-raw
option in curl IIRC.
Thanks
Hello i am testing a maven project,where Clojure code is also inside the maven. It works fine , but i want to use a class from Clojure code gen-class , and it said it cant find it
you can use https://github.com/talios/clojure-maven-plugin for that
it hooks into the compile phase to compile clojure code
you'll have to configure your pom to make that happen
the readme has more info
anyone know of a good transducer that kinda sorts? Ideally holds onto the largest N
items by some metric?
(transduce (biggest 5 identity) conj [] (range 30)
would build up a collection no more than five at any point in time as it reduced the collection
you could probably use https://github.com/cgrand/xforms window transducers
yeah i'm not familiar with his excellent library so i was hoping it would ring a bell as something already done with a proper immutable min heap
lots of other useful things there too
I need a TCP client library. I just want to send bytes and receive bytes. Aleph has exactly the kind of interface I'm looking for:
(def client @(aleph.tcp/client {:host host :port port}))
(manifold.stream/put! client (msg->bytes mx))
(bytes->str @(manifold.stream/take! client))
but I'm wondering if there are any lighter weight libraries out there (or even a gist?)sockets appear to form part of the https://github.com/clj-commons/byte-streams abstraction. Not very familiar with it though
the java socket stuff is pretty easy to use via interop, you don't really need a library for it
on one side you mostly just create a ServerSocket and .accept on it, which gives you a connection, which has an input/output stream on the other, create a new socket and use its input/output stream
pretty basic example: https://www.baeldung.com/a-guide-to-java-sockets
cool I'll give it a shot with sockets. I've used them a bit before but this time I got hung up on some (java) example code using Nio socketchannels and was trying to do it in that way which maybe has a bit more of a learning curve.
that's a couple more steps (but opens some new options)
I'm writing a clj wrapper for https://bitbucket.org/chromiumembedded/cef/ (cef) and am trying to figure out the best way to make it consumable as a library. The cef framework by itself is 80MB compressed and ~220MB uncompressed which seems too large to include in a library jar. Currently, I'm putting everything except the cef framework in the library jar and including a function that will download/extract the framework if it doesn't exist on the local file system. Is there a better way to handle this?
This will maybe also allow your tool to upgrade CEF itself without upgrading the jar, might add some flexibility.
Or if users already have it, they could maybe set some env variable to indicate where it is
supporting env variables would make a lot of sense. Currently, I only allow the consumer to specify the target dir, but supporting env variables at some point would also be good.
was asking about a biggest-by
transducer earlier but realized i needed a reducing function for this actually. Does anything about this stand out as problematic?
(fn queue-accumulator
([] (PriorityQueue. 30 cmp)) ;; custom comparator
([^PriorityQueue q]
(loop [acc []]
(if-let [x (.poll q)]
(recur (conj acc x))
acc)))
([^PriorityQueue q item]
(if (>= (.size q) threshold)
(let [smallest (.peek q)]
(if (pos? (.compare cmp item smallest)) ;; custom comparator
(doto q
(.poll)
(.offer item))
q))
(doto q
(.offer item)))))
What are you trying to achieve exactly? Find the biggest item in a collection by some definition?
Keeping only the biggest n items in a reducible collection. Imagine it’s a million rows from a db each with a score. Want to keep the largest scored values without requiring the entire result set and sorting to get the largest items
So are you OK witn O(N) performance? If that’s the case, perhaps a simple (filter …) transducer ?
i don't know how i could filter. given that i don't know a priori what some threshold will be
just that i want the largest N items. i don't have any fore knowledge of how to determine those N items until i pass a function over them
i certainly can't beat O(N) in the general case. i think i'll hit O(n log(n)) in the worst case scenario where its essentially just heap sort but i'm discarding values so i have much smaller memory footprint
but that's the point of using a heap here. i don't want to track order or sorting, just a collection of items with constant time access to the min so i can throw that one away if necessary
correct. that's why it needs to be in the reducing function to manipulate the concrete collection rather than in a transducer
or continually resorts a collection. you're essentially using it as a heap whereas i'm using the proper heap datastructure here. constant access to smallest item, no sorting of the rest of the collection
Is there a way to get the data sorted out of the database? Or that’s not a good option?
no. scoring things in code based on the rows from the db. far too complicated to implement that in sql
Have you seen this? https://github.com/clojure/data.priority-map
ah, it was ordering. it maintains the priorities to the items but doesn't expose a way to quickly get the lowest priority item
(require '[net.cgrand.xforms :as x])
(let [data (repeatedly 10000 (fn [] {:score (rand-int 1000)}))]
(sequence
(comp
(x/sort-by :score >)
(take 3))
data))
One attempt using the xforms library… I am not 100% sure if x/sort-by realizes the whole collection…. It might.yeah it does. it just keeps all of your items in an array list and on the completing single arity of a transducer it calls (java.util.Collections/sort cmp)
(defn sort
([] (sort compare))
([cmp]
(fn [rf]
(let [buf #?(:clj (java.util.ArrayList.) :cljs #js [])]
(fn
([] (rf))
([acc] (rf (core/reduce rf acc (doto buf #?(:clj (java.util.Collections/sort cmp) :cljs (.sort cmp))))))
([acc x] (#?(:clj .add :cljs .push) buf x) acc))))))
Depending on the database you’re using, I would still try to get all the items in a sorted order first… That would actually prevent you from having to go through the 1 million items in the first place (even if you’re not keeping them all in memory).
need rows and then run functions over the different columns using different ways to rank them
well doing things like longest common subsequence between search term and the columns doesn't sound fun in sql
but i appreciate you going through this with me. for sure i'd prefer a native clojure immutable heap or something similar
I am just suggesting asking the DB to do the sorting… I 100% agree about not doing most things in SQL, it would be a mess.
the values they sort by are computed dynamically according to search term. no way to ask the db to sort it
@dpsutton did you find a solution you were happy with for this? ... maybe something like this?
(defn keep-group [f init]
(fn [rf]
(let [g (volatile! init)]
(fn
([]
(rf))
([r]
(rf (reduce rf r @g)))
([r i]
(vswap! g f i)
r)))))
(defn largest-n [n]
#(let [group (conj %1 %2)] (if (< n (count group)) (disj group (first group)) group)))
(comment
(into [] (keep-group (largest-n 5) (sorted-set)) (range 10))
)
yeah. and it doesn't make much sense to do so much accumulation in the transducer. it clogs the whole pipeline until you're done which feels a bit presumptuous of a transducer
also there's no comparisons in yours. so it would just be a huge volatile in a transducer chain
it's in the largest-n
fn ... which limits the number of things in the volatile to n things ... right??
it's doing the comparison by putting the data into a sorted set (which I guess could be a sorted-set-by
... and it's dropping the first one from the set if the count is larger than the n
we give to largest-n
... so there's only 5 things in the volatile ... but in the same way you can't return the first result till you've reached the end when sorting, you can't do that here ... right?