Fork me on GitHub
#clojure
<
2018-09-10
>
vemv00:09:23

Can I concisely take N elements from a list, each having a distinct value according to fn f? e.g. (distinct-n-by-f coll 2 :age) ;; gives me 2 elements, and each will have a different :age

tristefigure00:09:30

first, map f onto coll. (map f coll). Then take 2 elements from it. (take 2 (map f coll))

vemv00:09:24

this forgets the uniqueness requirement (which I didn't make clear enough - sorry!)

tristefigure00:09:40

(take 2 (distinct (map f coll)))

tristefigure00:09:09

take, distinct, map : they all compose lazily which means you'll only compute what's needed for those two elements

noisesmith00:09:31

(comp (partial take n) distinct) ?

dpsutton00:09:54

Distinct has a transducer arity right?

noisesmith00:09:13

yes - I wasn't trying to make a transducer there but it is an option

noisesmith00:09:57

something like (into [] (comp (map f) (distinct) (take n)) coll)

noisesmith00:09:53

or do you need the original items that were distinct by f, and not f of the items?

vemv00:09:30

the former

andy.fingerhut00:09:23

Looks like the medley library has a function distinct-by that lets you supply f to it, but keeps the original items when there f values are distinct: https://crossclj.info/fun/medley.core/distinct-by.html

vemv00:09:46

What I really had in mind (and sorry for the confusion) is to "sort by distinctness" so to call it... e.g. given [1 1 1 2 2 3 3], return [1 2 3 1 2 3 1]

vemv00:09:04

I don't really need to have it implemented for me, can eventually do myself. Although a pointer to prior art would be welcome

vemv00:09:58

Possible simple solution: group-by and take one element from each group at a time

noisesmith00:09:07

here's one approach

noisesmith00:09:17

(defn distinct-by
  [f coll]
  (letfn [(step [seen [e :as xs]]
            (lazy-seq
             (let [match (f e)
                   s (seq xs)]
               (when s
                 (if (contains? seen match)
                   (step seen (rest s))
                   (cons e
                         (step (conj seen match)
                               (rest s))))))))]
    (step #{} coll)))

noisesmith00:09:20

based on distinct

noisesmith00:09:05

but sure, (comp (partial map first) keys (partial group-by f)) is another approach

✌️ 4
Denis G08:09:34

Are Clojure Agents also called Agents (not talking about AI thing) in academia, or CS world. Found this article http://wiki.c2.com/?ActorVsAgent and am trying to understand what is written there, and under their definition of agent → communicate to an agent by modifying something that it is observing; you do not need to know that the agent exists What does this even mean, do not need to know that the agent exists? Are we talking about the same thing?

henrik08:09:43

@denisgrebennicov They probably mean that the agent doing the modification doesn't need to know if anyone cares about the thing modified or not.

Denis G08:09:45

@henrik but we indeed are talking about the same thing. Meaning their definition of agent and definition on the http://clojure.org website are describing the same thing, right?

henrik08:09:52

It would be really out of character for things in Clojure to use arbitrary nomenclature. But I know too little about either to answer for sure.

Denis G08:09:36

Well for example refs is just a Clojure concept, right? Refs or references are known to be something else in other programming languages, or am I wrong?

deliciousowl10:09:42

What's the stage of AI in clojure? Probably will have to fall back on Python, or?

deliciousowl10:09:18

advanced neural nets to be specific (convolutional, generative ,etc)

deliciousowl10:09:37

" The MXNet framework is known for its great scalability" oh nice

deliciousowl10:09:34

Neanderthal is awesome, but then you have to do everything manually and that would take forever

jsa-aerial23:09:28

There is https://github.com/aria42/flare (seq analysis with RNN) and clj-autograd on top of Neanderthal. Also, this sort of question is likely to get more responses in #data-science

blueberry17:09:11

not at all. neanderthal does everything automatically, and you do not have to do (almost) nothing manually in the scope of linear algebra and vectorization. you may have to do things on your own when it comes to deep learning or something else, but neanderthal is not a deep learning library.

henrik13:09:57

I would like to do something that’s probably quite odd: run a functions juuuust before something is taken off a core.async channel. So let’s say my producer offers a value. The consumer might come along to request a value later, maybe much later, so the thread blocks until this happens. In the meantime, the value might have expired, in which case I want to offer a different value rather than the one that was initially on offer, thus applying a function to the value just before it is consumed. I tried to do this with (chan buffer (map func)), but for all its laziness, map seems very eager to run that function straight away rather than waiting for it to be realized.

rutledgepaulv13:09:32

I’m not sure there’s a way to do that in a transducer since it’s going to be unaware of the consumer showing up (afaik). You might have to just wrap the place of consumption so that you first poll the channel and drop its value if it’s not what you want. Maybe you could hide that behavior in a pipeline between two channels?

rutledgepaulv13:09:39

or maybe you could do it with a custom buffer? if a better replacement value shows up before you take from the buffer you could swap the value in the buffer?

👍 4
rutledgepaulv13:09:47

but sounds like maybe you don’t have a “better replacement value” showing up. you just want to compute one at consumption? Maybe you could reorganize that to just put potential “refreshes” on the channel too and then use the custom buffer

arnaud_bos13:09:52

Is Rich's paper "The Clojure programming language" available somewhere?

Alex Miller (Clojure team)15:09:41

I don’t think that is a thing

Alex Miller (Clojure team)15:09:24

if you are seeing it as a reference somewhere, it probably literally means “the Clojure programming language”

arnaud_bos16:09:43

I've found it as reference in a paper today and also listed on ACM here https://dl.acm.org/citation.cfm?id=1408682 But it is actually a reference to a conference talk... ResearchGate has it in the right category. I admit it got me curious.

henrik13:09:14

@rutledgepaulv Custom buffer is interesting, but yes, I have to execute code. An alternative I’ve thought of is that the consumer has to pass a return-channel to the producer over another channel. That way, the producer listens for incoming channels over the channel, and has the opportunity to produce the value fresh straight away, then put it on the return-channel that the consumer supplied. The reason I wanted to see if there’s another option is that this introduces at least two more channels, and I wanted to keep it more straightforward than that.

rutledgepaulv13:09:08

yeah. at a high level it sounds like you’re asking for a custom buffer to me. something that knows when values that have already been collected should be discarded and replaced with other values (however those arrive)

rutledgepaulv13:09:49

not sure how feasible that is without knowing more details of expiry / new values, etc

henrik14:09:49

The general problem could be illustrated like this:

(defonce db-ch (chan))
(defonce db (atom nil))
(def reset-db? (atom true))

(go-loop []
  (when @reset-db?
    (reset! db (get-new-db))
    (reset! reset-db? false))
  (>! db-ch @db)
  (recur))

henrik14:09:28

So, the DB connection needs to be refreshed. With this model, it only gets refreshed after the next value is taken off the db-ch.

rutledgepaulv14:09:20

just attach a watch to the atom that writes to the chan? and a sliding buffer on db-chan so it discards old values if unused

henrik14:09:37

Now, there’s a thought.

henrik14:09:18

With a sliding buffer, will the loop still block, or will it go nuts throwing ever newer DBs into the void?

rutledgepaulv14:09:36

it’d go nuts if there’s no other reason to block

rutledgepaulv14:09:34

ideally you’d just block on whatever is deciding that the db needs to be reset. or if you just wanted a periodic thing you could use a timeout

rutledgepaulv14:09:08

but I think you could remove the loop with just a watch on the atom and use >!!

henrik14:09:19

In this case, let’s say the DB may be requested many times between the time it is created and the time it is refreshed. So it has to keep supplying it to anyone who asks.

rutledgepaulv14:09:34

that sounds like a weird use for a channel. channels are more for data in motion than resupplying the same data to people who ask until it changes (imo)

rutledgepaulv14:09:12

why can’t the consumers just deref db in that case?

henrik14:09:37

Yeah, it might be the wrong tool. The channel acts as a choke point, essentially, to manage the lifecycle of the DB.

rutledgepaulv14:09:15

is it important that once it’s refreshed, all consumers are immediately told about it? or just next time they ask?

henrik14:09:21

Derefing is fine, but only one request to update the DB atom is allowed at any point in time.

henrik14:09:36

They need to be told immediately.

henrik14:09:46

As of the next value requested.

rutledgepaulv14:09:44

if they’re asking, then yeah I wonder if just an atom by itself is okay without core.async. by using a watch on reset-db that only runs the reset! when the value of reset-db? transitions from false to true it should be atomic since watches run after the CAS

sparkofreason16:09:23

What might cause the exception java.lang.IncompatibleClassChangeError: Implementing class when importing a class created with gen-class?

ghadi16:09:07

ICCE gets thrown by the JVM under many bad linkage conditions... like calling a static method using a instance method bytecode/descriptor

ghadi16:09:24

@dave.dixon got a sharable reproducing case?

ghadi16:09:17

can you dump the output of javap on the compiled class? It should match http://kafka.apache.org/20/javadoc/org/apache/kafka/connect/connector/Connector.html

ghadi16:09:21

@dave.dixon you also need to put classes on your :paths -- it's under the test alias right now

hiredman16:09:32

does the exception not give you the name of the offending class?

sparkofreason16:09:50

The exception does not give the class name. Added classes to :paths, same result. Output of javap looks correct AFAICT.

ghadi16:09:51

can you dump the output of javap pls

ghadi16:09:31

looks like you have two constructors with the same signature

sparkofreason17:09:04

:constructors was not specified correctly. Fixed that, now only one constructor, same error on import.

ghadi17:09:22

do you have the full trace?

ghadi17:09:27

(gist, please)

hiredman17:09:38

have you tried removing the gen-class from the ns form?

sparkofreason17:09:27

Yes. Same result.

sparkofreason17:09:11

Tried just using Object in :implements, same error on import.

sparkofreason19:09:55

Turns out the problem is that I needed to use :extends to implement an abstract class, not :implements, which is only for interfaces.

ghadi19:09:51

ah -- yeah!

ghadi20:09:20

you're going to have a bad time with setAccessible -- stuff like that will be denied in future JVMs

sparkofreason20:09:59

I was hung up on the "reference" docs at https://clojure.org/reference/compilation. When I went to the actual reference for the API, I found :exposes which is what I really wanted to get at a protected field.

sparkofreason20:09:36

It's a little ugly that gen-class will happily output a class that won't compile.

grav18:09:51

So nice to see both Calva and vscode-parinfer being actively developed on 🙂

👍 4
Karol Wójcik18:09:05

Is there a way to create a new Collection from Collections$UnmodifiableCollection?

hiredman18:09:08

lots of collections have a constructor that takes a collection

Karol Wójcik19:09:29

Got it thank you 😛

blueberry17:09:11

not at all. neanderthal does everything automatically, and you do not have to do (almost) nothing manually in the scope of linear algebra and vectorization. you may have to do things on your own when it comes to deep learning or something else, but neanderthal is not a deep learning library.