Fork me on GitHub
#clojure
<
2017-09-01
>
kenny00:09:04

Yeah that's what I thought.

dadair02:09:27

Is there a library for kafka with a core.async facade that works with kafka 0.11.0.0? kinsky seems to be broken for >= 0.10.1.0

donyorm04:09:22

Ok I'm working with macros seriously for the first time. I have this body

`(~(edn/read-string f) ~@args)
, where f is an argument. However I need f to be evaluated, not just a symbol or a list (in case I want to pass, say (str g) as f). How can I do this in a macro? Right now, I'm getting a class cast excpetion when I try to run the macro

didibus08:09:20

Not sure you can without eval

didibus08:09:27

(defmacro run
  [f & args]
  `(~(edn/read-string (eval f)) ~@args))
(macroexpand '(run (str 'Math/max) 120 12))

didibus08:09:38

The reason is because this does not work:

((edn/read-string (str 'Math/max)) 120 12)

didibus08:09:18

You really need the macro to return the result of read-string at compile time

didibus08:09:52

That's only because of the interop

didibus08:09:30

Otherwise this would work:

((resolve (edn/read-string (str '+))) 120 12)

didibus08:09:08

And you could do:

(defmacro run
  [f & args]
  `((resolve (edn/read-string ~f)) ~@args))
(macroexpand '(run (str '+) 120 12))

jeff.terrell12:09:33

@U1C03090C - f will be evaluated automatically. Macros return an expression, and that expression is automatically evaluated, so since f is part of the expansion, it will be evaluated. I'm guessing your class cast exception might be something else. Is args a sequence?

jeff.terrell12:09:21

I once wrote a blog post that has some good tips for writing macros and helps explain some of these concepts. Unfortunately, the blog is currently offline. (I'm hoping to get it online again soon.) In the meantime, I'll attach the content of the post for your reading pleasure. simple_smile

Nick Drew09:09:35

How would you go about 'securing' a value in a data structure so that it would masked whenever printed. But when passed into the destination function (a java method) it was unmasked

Nick Drew09:09:17

E.g a 'secure-string' wrapper

dominicm09:09:19

you could override the printer multimethods

Nick Drew10:09:26

@dominicm that's a good steer. Is it possible to auto unbox as well?

pseud10:09:44

Anyone actively using Kafka ? I was wondering, the Apache Kafka site lists 2 GH projects for Kafka bindings (https://cwiki.apache.org/confluence/display/KAFKA/Clients#Clients-Clojure.1) though one is has a failing build & no change in 2 years while the other has had no changes in 5(!) years. Are clojure people not using Kafka or is using the Java interop good enough(tm)?

mpenet10:09:45

I think kinsky and gregor are the 2 most up to date

mpenet10:09:54

not sure about the ones mentioned on the wiki

danm11:09:29

We are using Kafka, but tbh we've not bothered with any of the clojure libs. We're just calling the Java stuff directly and have our own very thin shim over the top to, for e.g. turn a consumer into a lazy sequence of messages our code can read from

hmaurer11:09:06

Is there a special naming convention for functions in clojure that throw an error instead of returning nil?

hmaurer11:09:35

For example, I could have a function which finds a database record by ID and returns nil if it does not exist, and another function which throws an error if it does not exist

chrisjd11:09:38

I would use a ! suffix, personally.

leonoel12:09:12

all functions can throw an error instead of returning something

joshjones13:09:02

@pseud we're doing the same as @carr0t , wrapping the Java client. I'd be cautious about combining intrinsically I/O kafka with lazy sequences though, just my opinion

danm13:09:46

Works quite well in our case. Avoids the needs for loops etc, and just stops doing anything when we have no messages to consume

danm13:09:12

We do have to be careful that the lazy sequence is actually consumed when the output is side-effects, but we've got a standard pattern for that

dadair15:09:44

Kinsky does not work with Kafka 10.1.0 or later, I ended up just using the java interop directly, works ok for general cases

pesterhazy16:09:42

Java interop is often the best option, even if clojure wrappers exist

noisesmith16:09:17

yeah, I often have the feeling people are using a clojure wrapping not for pragmatic reasons but for aesthetic or sentimental ones

noisesmith16:09:34

and that they could save a lot of trouble by using the java API for the thing directly

gonewest81817:09:56

Which is the canonical github repo for clojure-protobuf? Is it ninjudd/clojure-protobuf, or flatland/, or ghaskins/, or some other?

seancorfield18:09:01

@gonewest818 flatland says it is forked from ninjudd

seancorfield18:09:26

And so does ghaskins.

seancorfield18:09:52

(GitHub shows forked from at the top of the page under the repo)

gonewest81818:09:13

@seancorfield right, however those forks look more active, as if maybe ninjudd has moved on to other things.

seancorfield19:09:44

Have you checked Clojars to see where the most recent builds are coming from? (I haven't looked yet, but that's where I'd go next)

seancorfield19:09:36

Wow, that's probably even more confusing...

seancorfield19:09:22

The only version I see there more recent than the one ninjudd pushed (which is under org.flatland!) is https://clojars.org/org.clojars.brunchboy/protobuf

seancorfield19:09:32

And that's July 2015.

seancorfield19:09:08

Despite that, it does look like ghaskins version is the most up-to-date ... so, yeah, it does look like ninjudd/flatland's version is way old. So I guess it'll depend on which version of Google protobuf you want under the hood since ghaskins' version is much more recent...

ghaskins19:09:16

@seancorfield @gonewest818 honestly, the whole state of protobufs on jvm is indeed a bit of a mess

ghaskins19:09:30

but fwiw, I do use my fork in production on the hyperledger project

ghaskins19:09:39

IIRC, there were two main things I did from the ninjudd base: 1) I got rid of the implicit ties to a specific protobuf tooling, and 2) I got rid of attempts to compile protoc and auto-protoc compile

ghaskins19:09:59

and of course, I also bumped up the protobuf support to more recent versions

ghaskins19:09:21

(1) and (2) were causing me lots of problems

ghaskins19:09:24

That said, using cljs+protobufjs on nodejs is much more pleasant than trying to use clj+jvm

ghaskins19:09:46

anyway, would be happy to cut a v3.4.0 based release for you guys if you needed it

ghaskins19:09:00

ive been meaning to upgrade my projects anyway

gonewest81819:09:27

Wow, thanks for all the responses... I was at the hardware store picking up odds and ends, and besides I'm on vacation right now so it isn't super urgent but yeah, clarifying the situation for clj+jvm seems like a step forward for the community. At very least, having an active project with more obvious namespace correlation between github and clojars would save people the head scratching.

seancorfield20:09:49

It would be nice if Clojars had ways to sort the list of projects by date/version/group ID and maybe a "refine search" option too. Clojars is open source so anyone could send in PRs for that sort of stuff.

seancorfield21:09:06

It can be very hard to figure out whether the original GH project has been abandoned -- and sometimes, just because a fork has more recent commits, it doesn't mean it's a good, safe, solid update on the original (and it can take a while for a fork to build up a good head of "downloads" steam). Definitely looks like yours should be the go-to choice tho' now @ghaskins -- 💯 for giving the project new life!

seancorfield21:09:54

I forked an old Axis wrapper Clojure project to see if I could get it working with Clojure 1.3. I did, but in the end it didn't do enough for me and I switched to raw Axis via interop. But I got pinged for help with Axis for years after forking that project! Lots of people looking for a better solution but no one seemingly able to create one (me included).

ghaskins22:09:34

@seancorfield I know what you mean: I went looking for protobuf integration years ago and the landscape was a mess and confusing (old, didn't work, etc). The only thing that ended up working for me was the fork+patch we are discussing now. To your point, I added extra entropy to the messy landscape with my approach, but it does seem to work and thus satisfied my initial itch. I would be willing to try to clean up namespaces, etc, but I didn't want to step on the toes/attribution of the original contributors, thus the retained flatlands/ninjudd namespaces etc. Open to suggestion for the best way forward so the community has a robust option, though.