Fork me on GitHub
#clojure
<
2019-05-27
>
markx03:05:37

Is there a way to fan out a seq, turning a seq into multiple seqs?

markx03:05:11

It seems straight to me how to fanout a channel into multiple ones, but I’m not using async, so I wonder if there’s a way to achieve the same thing for seq.

markx03:05:48

My real use case is that, I’m parsing a stream of bytes from a socket with a telnet-like protocol. I want to separate the telnet command bytes from the text bytes.

didibus03:05:02

You could use juxt maybe? Or partition?

markx03:05:12

Currently what I’m doing is turning the bytes into a seq of packets, so something like

b b b b ... b -> {:type :telent :bytes [1 2 3 4]}, {:type :text :bytes [1 2 3 4]}, ...
Then I need to parse all the :text to parse out some bytes for another protocol. I guess I can re-assemble all the :text packets into a byte seq again, but I am afraid that there will be some performance issue.

didibus03:05:17

In what way?

didibus03:05:34

Seems in your case you could just do sort-by

didibus03:05:25

Sorry I mean group-by

markx03:05:33

well I don’t see a clear way to do this with group-by. To parse a byte and know which type it belongs to, I need to also check its previous and next bytes.

ahungry03:05:00

Using cljfx, is there a way to use openjfx Canvas? I can't find a sample on the site, and it isn't immediately intuitive as to how I would do it (or even make a plain OpenJFX element and add it to a cljfx node)

didibus03:05:43

Oh, sorry I thought your example was showing the datastructure and you had them already parsed into maps

markx04:05:56

I do have them already into two maps, but this doesn’t work well if I want to parse the bytes of the third type.

didibus04:05:01

You need to parse the byte array in the map? Or across them?

isaac04:05:44

IMO, Should considering TreeMap, TreeSet, PersistentTreeMap on fn last implementation

nielsk08:05:27

@markx So how do you want to groupo your bytes ? By a fixed number ?

markx14:05:37

It’s a telnet like protocol, so the sequences are identified by certain byte patterns. For example 255 is used as an escape character, so 255 249 means go ahead, and 255 255 means 255, kinda like escaped 255, and etc.

markx14:05:43

And so they are not grouped by a fixed length.

nielsk08:05:43

So 4 bytes apiece ?

dominicm10:05:17

Has anyone experimented with grpc/protobuf and deps.edn? Is there a good library for compiling protobufs?

mirza1b3911:05:39

google protobuf java library?

mirza1b3911:05:11

just generate java code from protobuf and use it with java interop thats how i do it with kafka

henrik11:05:49

I'm running some compute intensive calculations on my Macbook like an idiot, when I have a monster of a gaming PC sitting in the next room twiddling its thumbs. I though maybe I could kick up a REPL on it and outsource the job.

dominicm12:05:49

It does, yeah

👍 4
carkh04:05:27

also works just as well on just windows

carkh04:05:56

without the IO tax of WSL

henrik09:05:04

Does it now? Maybe I'll just try that straight up then.

henrik09:05:57

For some reason I thought clj wasn't stable under Windows native.

carkh09:05:00

been using clojure on windows since beta, making artifacts that would later run on solaris etc

👍 4
carkh09:05:46

the newish clj tool has trouble on windows yes, i have a workaround for that... let me find it

carkh09:05:23

so, follow the procedure outlined on the clojrue web site to install it on windows

carkh09:05:33

then in your project root put this file

carkh09:05:12

it will launch clj from a normal command prompt

carkh09:05:29

usefull for some tools that go around powershell

carkh09:05:45

anyways depending on your workflow you might not even need it

carkh09:05:56

if you're not using deps.edn, no need

henrik09:05:51

Thanks, I'll try this. I use deps.edn exclusively, so I do need it.

henrik09:05:05

(Datomic Cloud)

carkh09:05:15

have fun !

henrik10:05:28

@U13RRNF2M For connecting, I'm getting timeouts on the REPL connections (so the REPL server isn't answering, I'm guessing), even though I've added rules for both ingoing and outgoing TCP connections on the correct port in Windows Firewall. Do you happen to know if I would need to do anything else to open the ports?

carkh10:05:07

you're connecting to a remote repl ?

henrik11:05:06

Yeah, but I'm an idiot. I forgot the configure :host and :bind for nREPL.

henrik11:05:13

(They default to localhost)

carkh11:05:36

hah ! anyways i never had to go remote like this so i couldn't help

henrik11:05:20

Yeah, I'm setting it up so that I can essentially use my laptop as a thin client to a much more powerful machine. But the laptop is still nicer to work on 🙂

lvh13:05:43

I have a project that has a sever side component and a client side component. they have different dependencies, and I'm keeping them separate via lein profiles. For development convenience I want to merge the two, and I tried :repl [:client :server] in my project.clj. However, while lein repl does not include server-only or client-only deps, lein with-profile +repl repl does. Am I misunderstanding hwo the repl profile works? Do I add it somewhere else instead? Am I misunderstanding how lein repl works?

rickmoynihan14:05:53

I’d add them to :dev profile, which :repl should also source

rickmoynihan14:05:33

:dev will also make the deps available to your tests.

lvh14:05:55

awesome, thanks! sorry, I just figured this out and then read your message 😄

lvh14:05:01

(and yes that does appear to work)

dpsutton13:05:32

https://github.com/technomancy/leiningen/blob/master/doc/PROFILES.md#task-specific-profiles seems to confirm your expectation. > Some tasks automatically merge a profile if specified. Examples of these are the :test profile, when running the test task, and the :repl profile, when running the repl task.

lvh14:05:32

putting it in the :dev profile appears to work

dpsutton14:05:09

that might be worth raising an issue. the code appears to try to get the repl profiles.

dpsutton14:05:28

i think it doesn't track down the transitive profiles there

markx14:05:37

It’s a telnet like protocol, so the sequences are identified by certain byte patterns. For example 255 is used as an escape character, so 255 249 means go ahead, and 255 255 means 255, kinda like escaped 255, and etc.

ghadi14:05:14

@dominicm I've been working a side project to do protobufs without Java interop

dominicm14:05:39

@ghadi on my god please yes

ghadi14:05:34

byte array -> Clojure map -> byte array

ghadi14:05:00

the wire protocol isn't that tricky

dominicm14:05:22

My experience with protobuf (and grpc) was not great. I spent an hour on it, and I didn't get anywhere.

dominicm14:05:37

I know that's not much, but it was incidental to what I wanted to do.

ghadi14:05:44

Yeah it would be really cool if we could participate more smoothly in that ecosystem

lvh14:05:14

@ghadi neat! what's that look like -- gloss?

lvh14:05:26

(I'm interested in that as well because GCP)

dominicm14:05:43

That would be great.

dominicm14:05:17

Time to make sure I'm following you on github

ghadi14:05:17

read the .proto definitions with instaparse and do some analysis on them, turning each message into a encoder/decoder

ghadi14:05:29

I haven't published it yet

ghadi14:05:48

but I haven't made progress on it in a couple months, so maybe I should just push the code up

ghadi14:05:12

the output of the analysis phase should be plain data so that you could write it down and skip analysis if desired

ghadi14:05:28

kinda like cognitect aws-api spits out .edn descriptors of AWS api's

lvh15:05:42

yes please, my thing consumes that data and only incidentally the apis themselves

Jakub Holý (HolyJak)15:05:12

Does anyone know why https://clojure.github.io/clojure/clojure.core-api.html#clojure.core/gen-class does not mention that it is possible to attach Java annotations to methods and constructors? @alexmiller? (Neither does the Java Interop page, but that is not so surprising)

seancorfield18:05:40

Probably because you can attach annotations to code without using gen-class?

seancorfield18:05:24

I know that's not adding annotations to methods/constructors, just saying that you can do that in the general case outside of gen-class.

seancorfield18:05:40

Is the syntax much different for doing this within gen-class?

seancorfield18:05:17

(and, yeah, it would be nice if adding annotations was better documented somewhere...)

👍 4
Jakub Holý (HolyJak)22:05:29

thx, I had no idea! And I'd correct >, it would be nice if adding annotations was better documented somewhere... to > it would be nice if adding annotations was documented somewhere... :)

seancorfield22:05:08

Hah... yeah. I thought they were documented somewhere but I remember it being hard to find anything about it.

phil16:05:50

I'm having an issue in a large app where a top-level expression is causing a block on startup. Is there some flag I can pass to something to see what point clojure is at when it stops?

ghadi17:05:17

@phil start clojure with clojure.main and manually require your code entry point namespace with (require 'your.code :verbose)

👍 16
mpenet18:05:44

oh! :verboseis new to me, nice

borkdude19:05:59

if you write 'foo/bar or

`foo/bar
or
(quote foo/bar)
how should that affect linting required but unused namespaces? In Clojure, you do not need to require namespace foo in either case, although joker counts foo as used in the case of the syntax-quote. Currently clj-kondo does the same for syntax-quote and quote, although it probably shouldn’t in all these three cases.

vemv21:05:20

I think it's a good idea to lint

`foo/bar
as a fault if the foo alias does not exist in the current ns. Else:
`foo/bar

vemv21:05:46

will likely evaluate to sth different than what the programmer intended

vemv21:05:35

I would distinguish syntax-quote from plain quote though: only the former has the alias-expanding semantic, so I would only lint for the former

borkdude21:05:02

I have made two comments here: https://github.com/borkdude/clj-kondo/issues/210#issuecomment-496293314 I think that’s the same as what you’re saying here right?

borkdude21:05:55

so syntax-quote handles the symbol as “using” the namespace, but normal quote doesn’t

vemv21:05:58

I agree if in: > but these do not give the warning, which I believe is the incorrect behavior. "the warning" means a warning other than is required but never used, possibly a new one

leontalbot20:05:47

Hi! Say I want the 4 fastest non-nils answers from 10 concurrent api calls, what is a good and fast way to do it?

(def ids `[1 2 3 4 5 6 7 8 9 10])
(defn ask 
  "may return nil"
  [id] 
  (:body (http/GET {...}))
(defn take-4-first-non-nils [ids] ...)

vemv21:05:42

You can play with pmap and a blocking queue. Each fn in the pmap pushes to the blocking queue. Then you grab 4 elements from the queue, which by definition will be the fastest ones Should be simpler than core.async code

vemv21:05:34

(just beware that grabbing 4 elements may block forever if done incorrectly)

souenzzo20:05:36

@leontalbot you will need async/alt! from #core-async

leontalbot20:05:15

@souenzzo Thanks 🙂 Any example where we wait for first n results before returning them?

leontalbot20:05:18

I’ve always seen example with the first result, but not more

leontalbot20:05:03

thought it would start like so, but I don’t know actually (async/take 4 ...)

souenzzo20:05:24

Yeah. something like *the* first result, 4 times in a row 😅 Not sure about the answer but I'm sure that #core-async and alt will compose that answer

👍 4