Fork me on GitHub

Sorry on vacation. When I get back happy to chat but not really interested in private discussion about stuff like this. Against the point really. But feel free to reach out when I return.


i think the best thing about clojurescript is that they actively use #cljs-dev and you can follow along with the enthusiasm that the developers have to their field. you can follow along with how issues are approached and you can see how priorities are set. i think this is drastically different from the way clj development happens


I'm a noob to clojure but am working through books and examples and I am trying out the whole proxy, gen-class, interface, defrecord, deftype, reify thing fight now and it appears to me that you can use proxy to override Java object methods OR implement interface methods on an anonymous object but NOT BOTH. Is this correct?


@funkrider If you're new to Clojure, you'll probably find #beginners helpful for "getting started" with any particular feature. As for proxy, according to the docs, you can supply both a class name and interfaces -- but only one class (as many interfaces as you want).


@funkrider As a silly example:

user=> (def p (proxy [ clojure.lang.IFn] [] (read [] 123) (invoke [] "World!")))
user=> (.read p)
user=> (p)


(the latter is just shorthand for (.invoke p))


thanks sean! I get your example but why wouldn't this work? (defprotocol prot (foo [this])) (def p (proxy [ prot] [] (read [] 123) (foo [] "World!")))


@funkrider My first reaction is "a protocol is not an interface"...


(you get clojure.lang.Var cannot be cast to java.lang.Class, right?)


omg (palmface) interface, protocol not the same thing... Thanks sean 😉 And yes thats the error I get


You can reify protocols and interfaces (but not classes)

boot.user=> (def p (reify prot (foo [this] "Hello") clojure.lang.IFn (invoke [this] "World!")))
boot.user=> (foo p)
boot.user=> (p)


BTW @funkrider you can use triple-backtick to format blocks of code (feel free to ask in #slack-help for hints & tips on using Slack).


OK so - is there a way to extend a java class, such that the class has some new private state and either option A) a new method on the class instance that has access to the state or option B) a new clojure protocol can be created and dispatched based on the extended class type that has access to this new private state within an instance of the class? Essentially I am working through a bunch of interview type questions in Clojure to compare use against say Java and the question goes like this: Create a new Stack class that implements push, pop and max methods and all operate in O(1) time. It is relatively easy in Java but in Clojure I am really having a hard time getting it done!


are you doing this in clojure just to see how to do it in clojure? iow, is there a reason for the java interop?


The bottom line is really that "Java Class" is not the idiomatic way to do this in Clojure. So you wouldn't create a mutable object in the first place -- the question just doesn't make sense.


But, if you're determined to create some sort of immutable data structure with push, pop, and max, that return a new version of the data structure for push and pop, then you're looking at a protocol for the functions and a record for the state...


...sounds like you'd need a stack that contained pairs of max and item to retain O(1) performance?


(defprotocol PMaxStack (push [this item]) (pop [this]) (max [this]))
(defrecord MaxStack [stack] PMaxStack (push [this item] (if stack (let [[m i] (first stack) m' (clojure.core/max m item)] (->MaxStack (cons [m' item] stack))) (->MaxStack [[item item]]))) (pop [this] (->MaxStack (rest stack))) (max [this] (ffirst stack)))
(untested, off the top of my head)


Ah, you'd need to exclude push, pop, and max since they're in core -- or rename them.


Also notice you can't have pop both return a new MaxStack and the first item so you'd need some sort of peek accessor for that...


Or else say (second (first (:stack my-stack))) which is kind of opaque. Anyway, hope that helps @funkrider ?


Here's a working example:

boot.user=> (defprotocol PMaxStack (spush [this item]) (spop [this]) (smax [this]) (speek [this]))
boot.user=> (defrecord MaxStack [stack] PMaxStack (spush [this item] (if stack (let [[m i] (first stack) m' (clojure.core/max m item)] (->MaxStack (cons [m' item] stack))) (->MaxStack [[item item]]))) (spop [this] (->MaxStack (rest stack))) (smax [this] (ffirst stack)) (speek [this] (second (first stack))))
boot.user=> (->MaxStack nil)
#boot.user.MaxStack{:stack nil}
boot.user=> (spush *1 4)
#boot.user.MaxStack{:stack [[4 4]]}
boot.user=> (spush *1 2)
#boot.user.MaxStack{:stack ([4 2] [4 4])}
boot.user=> (spush *1 8)
#boot.user.MaxStack{:stack ([8 8] [4 2] [4 4])}
boot.user=> (smax *1)
boot.user=> (spop *2)
#boot.user.MaxStack{:stack ([4 2] [4 4])}
boot.user=> (speek *1)
boot.user=> (spop *2)
#boot.user.MaxStack{:stack ([4 4])}
boot.user=> (smax *1)
boot.user=> (speek *2)


and gen-class doesn't seem like the "Clojure" way especially as it ends up being tied to a specific clojure version?


Thanks sean for the tip on formatting too!


for testing purposes I'd like to use with-redefs to replace a function with one suitable for testing. however, the project uses direct linking, so I will have to define the function that can be redef'd with ^:redef -- is the best way to get this directive out for production to elide-meta when i compile? or is there a much better way to do this?


preferably, a way to avoid direct linking when i'm running tests so i can avoid ^:redef altogether?


I'm writing a webapp where both server and client uses clj/cljs. I'm currently serializing/unserializing using str and edn/read-string. Is transit a drop in for this? Due to websockets, it wants / givbes me strings. However, transit seems to want to control it's own streams.


“transit seems to want to control its own streams” is, to me, a really odd way to put it


it’s simple to make a string out of a stream, the stream is the simpler and more general thing, which is why it’s the level transit operates on (so people don’t need to go creating strings that would not need to exist, out of an InputStream they already have)


it’s not a drop-in, and you’re right, transit wants streams


but it’s not a pain in the ass


I’ve written this like 3 times, now, I should probably put it on clojars:


new to clojure here trying to make heads and tails of documentation and tutorials etc and a bit lost. Hoping to test clojure for data science purposes. Using the following projects: boot, huri, clojure-csv, semantic-csv, neaderthal, kixi, bayadera in a gorilla repl with emacs (spacemacs) as my editor. 1) I want to know if anyone can point me to good resources to get more comfortable with clojure in data science? 2) bayadera is struggling to install: I use [uncomplicate/bayadera "0.1.0-SNAPSHOT"] ,but get could not find artifact error, can anyone give me some advice/help with this?


(->> [1 2 3 4 5 6 7 8 9 10]
     (map #(do (println %) %))
     (take-while #(> 3 %)))
what’s the proper way to ensure that only 1 & 2 are printed?


Swap the map and the take-while?


@jeff.terrell the result of the map is important for the take-while predicate


@martinklepsch the short answer is, don't do it. Mixing I/O with lazy sequences is not a good idea. What are you trying to actually do?


You can "de-chunk" the sequence, but that's all a workaround that usually indicates you don't need to have this in a lazy sequence to begin with


@rauh @joshjones thanks, interesting points. I want to execute a potentially side effecting function f for each element in a sequence and abort if (f x) returns an “error” (error represented by a special keyword)


@martinklepsch I'd use transducers instead of dechunking


reduce + reduced is a good choice


;; condition to terminate is when input is 3, in this example
(reduce (fn [_ x]
          (if (= 3 x)
           (reduced x)
           (do (println "value: " x) x)))
        (range 20))


@joshjones yeah, thats a good suggestion, thanks


@pjones Welcome to Clojure! It lo ks like the author has not released that library yet -- I can't find it on -- so you'd have to clone the repo and build it yourself. Since you're new, #beginners will be a helpful channel for you. Also #boot since you're using that.


There’s also #data-science which might be a good channel to hang around in.


Is there a "best practice" for creating non-global core-async pipelines? As an example, suppose I have a trivial async computation pipeline that looks like this: `(def in-chan (chan)) (def out-chan (chan)) (pipeline 16 out-chan (map (juxt identity (partial * 2.0))) in-chan) (go-loop [] (prn (<! out-chan)) (recur)) (defn inject [query] (go (>! in-chan query)))`


it feels like a bad practice to pus put that right at the ns level.


(def in-chan (chan))
(def out-chan (chan))
(pipeline 16 out-chan (map (juxt identity (partial * 2.0))) in-chan)

(go-loop []
  (prn (<! out-chan))

(defn inject [query]
  (go (>! in-chan query)))


sorry, bad formatting first time


option B could be:

(def in-chan (chan))
(def out-chan (chan))
(pipeline 16 out-chan (map (juxt identity (partial * 2.0))) in-chan)

(defn launch []
  (go-loop []
    (prn (<! out-chan))
(defn inject [query]
  (go (>! in-chan query)))


you decide to initialize when ready, but the pipeline sits at the ns level. Seems better.


In order to make inject work in-chan needs to be in scope, so unless the whole thing is wrapped in a lambda and inject were returned as a fn I don't see how to encapsulate it all.


for long text you may consider the #core-async channel fyi you still have global stuff, unnecessarily i think -- no reason you can't create the chans in a function along with setting up the pipeline, and pass those channels to functions which do something with them


ah, didn't even know #core-async existed. I'll pop over there.


I would opt for simple first; if your solution works as it is, I would stick with it. Otherwise you’re complecting things by creating “context” objects for no real benefit.


YAGNI or whatever. 🙂


I would avoid globals and opt for arguments


launching the pipeline belongs in launch... as well as construction of the channels


you have an infinite loop in the consumption function @markbastian . Need to check (when-some [v (<! chan)] (prn v) (recur))


when the channel is closed it will return nil repeatedly and infinitely.


cool, thanks for the tip.


(defn sink [ch f]
  (go-loop []
    (when-some [v (<! ch)]
      (f v)

(sink ch prn)


@tbaldridge knows that function ^


hey i want to ask you about experience with grouping logs. For example in sentry there is fingerprint conception . It works like ['foo' 'bar'] is grouping with others logs with the same values, but not with only ['foo']. Do you use it? What method do you have to group logs?


@markbastian I agree with @ghadi. Taking channels as arguments (instead of storing them in vars) is pretty essential to writing compose-able software.


Thanks for the tips. I figured that was the case, but I haven't seen any core-async resources online that explain the "right" way to use them. This is really helpful.


you're welcome mark -- generally you'll follow principles similar to what you'd use in any situation, i.e., prefer local state / params to global state


How do I get all the methods that are part of a protocol?


And hello everyone 🙂


I would like to create local bindings for all functions of a protocol against a certain instance of a record. So I want (to build) something that allows me to write (local-my-fn 123) instead of (protocol/my-fn impl/my-record-instance 123) for all my-fn in protocol/MyProtocol.


@seancorfield thnx, will definitely check that out 🙂


@chrisjd thnx, will add


On a different topic: I have a config.json, where I define which implementation shall be used for certain parts of my application. I start these parts using "mount", with (defstate my-part :start (condp = (:part config) "impl1" (impl1/->Part) "impl2" (impl2/->Part)). How would I do that without hardcoding all the implementations? Could I somehow make the different implementations for each Part register themselves? I played a bit with top-level forms in the different part's namespaces, which would update an atom somewhere, but that does not work, since the the code where the-atom-is-updated and where the-atom-is-used-for-lookup is not in the correct order.

Alex Miller (Clojure team)17:08:29

@urzds a protocol is a map that contains its implementation

Alex Miller (Clojure team)17:08:39

(defprotocol P (m1 [x y]) (m2 [a b]))
=> P
{:on user.P,
 :on-interface user.P,
 :sigs {:m1 {:name m1, :arglists ([x y]), :doc nil}, :m2 {:name m2, :arglists ([a b]), :doc nil}},
 :var #'user/P,
 :method-map {:m2 :m2, :m1 :m1},
 :method-builders {#'user/m1 #object[user$eval2043$fn__2044 0x7087e497 "user$eval2043$fn__2044@7087e497"],
                   #'user/m2 #object[user$eval2043$fn__2057 0xbbcec2d "user$eval2043$fn__2057@bbcec2d"]}}

Alex Miller (Clojure team)17:08:05

none of this is “public” api (you won’t find it in any docs), but it’s also not something I would expect to change


@alexmiller: So I could get all methods belonging to that interface by iterating over the :method-map? And with that list, I would write a macro, that generates the defines the local names / bindings?


Hi guys, what’s the “modern” way of building http apis these days? I’ve tried liberator and yada, but want to get more exposure


tried compojure also


Yada and liberator are both fine. pedestal is also a good option. It's all a matter of tradeoffs. Liberator and yada have a resource abstraction but the first binds you to the ring api, the later to Netty. Pedestal has solid async support but you need to implement a lot of the http resource semantics yourself.


I’d like something with swagger, component/mount/etc and validation out of the box. nothing wrong with ring or netty imho, as long as I get an uberjar


I’m a fan of compojure-api


the swagger and component integration has been great; we had to do a lot of finagling to support async responses, though


I would recommend looking for (or writing) a language designed to be sandboxed like that


clojure isn't, so it will take a lot of work to do it


basically you'll need what amounts to a custom compiler


the clojure compiler will emit code that uses reflection at runtime when the compiler doesn't know the classes involved, so not only do you need to control compilation, but you need to be able to do your checks at runtime


I presented a REPL for our Java services, and the team was uncomfortable with exposing dynamic code execution even in our pre-prod environment


I'm trying to find a good way to secure a REPL without requiring too much work


it's already behind HTTP security using drawbridge


@baritonehands Consider copying the socket REPL code in clojure.core.server and adding a TLS socket listener to it, and then maybe wrap clojure.main/repl with a simple auth mechanism (PSK)


nREPL is a whole lot of baggage


if its for trusted users, I wouldnt care about sandboxing - not that different from having access to prod db, otherwise I wouldn’t expose a repl. too many possibilities 😄


I built a system, where I can select which implementation of a Protocol to use based on a config file (using mount). Then I built multimethods to "register" these different implementations to certain config strings. Now the problem is that Clojure apparently never evaluates the namespaces where I define those multimethod implementations and hence I cannot load the Protocol implementations. Is there a better way to do such "component registration and selection at runtime" or is there a way to force Clojure to evaluate certain namespaces, even if they are not required from another namespace?


@mping Unfortunately that argument didn't work


@ghadi Has to be HTTP, so I don't have to expose any new ports


The problem with requiring the namespaces from the "core" namespace is that there would be cyclic dependencies, since those require the "core" namespace again.


@urzds Seems like you could require things dynamically, if you have the string of the full namespace


that was for reloading a namespace of an existing symbol, like my-ns/foo


@urzds this is pretty common, the problem is your multimethod/protocol definitions shouldn't be in your core namespace


they go in some other namespace


you multimethods/protocol defnitions are like interface definitions


@baritonehands That works:

(mount/defstate my-protocol
 :start (do
  (require (symbol (str "" (:my-protocol config))))
  (my-protocol/start config)))


do not do that, just structure your project well


I will, once I figure out how.


you defprotocols and defmulti's don't belong in your core namespace


hey, i am looking library like but for business logs purpose. Unfortunately i can’t use timbre, because i use it for developer logs purpose. But i needs more or less the same logic for business logs purpose. Do you recommend something? Just i want push to something some data and add appenders to do something with this data.


@urzds create new namespace, typically ending with protocols or spi, and stick defmultis and defprotocls in there


then the implementation requires that, and core requires that


(and core requires the implementations)


@hiredman: They are in The defrecords and defmethods are in namespaces. The problem appears when I try to create the state from the config using (mount/defstate ... :start ...). I would like the to register themselves, so I don't have to hardcode the list somewhere.


"the problem" being what I described above: The implementations of the multimethod not being know, since the namespace was never required anywhere.


@hiredman: Before I tried the multimethod approach to selecting implementations based on a config file, I had exactly the core -> impl -> protocol setup, but ran into some issues: my-ns.core requires my-ns.protocol1.impl1 so it can parse / execute the config file and create my-ns.core/protocol1-impl (using either my-ns.protocol1.impl1 or my-ns.protocol1.impl2). my-ns.protocol1.impl1 requires my-ns.protocol2 so it can use that, but it also requires my-ns.core, so it can access the selected implementation at my-ns.core/protocol2-impl. Thus there was a cyclic dependency. I could not break that cycle by moving (mount/defstate protocol2-impl ...) into my-ns.protocol2, because that would also have created a cycle between my-ns.protocol2 and my-ns.protocol2.implX. So I tried to break that cycle using the "everyone registers themselves" approach, which also breaks, because now my-ns.core requires nothing but my-ns.protocol1 and my-ns.protocol2, and hence no implementation is ever registered / defmethod is never called.


@kwladyka Could you not just use a different config with timbre for business logging? See the GitHub page you linked under “Configuration”; you could easily wrap the log* / logf* macros in a “business logging” ns, for instance.


thanks, looks promising, i have to find some examples


oh i see, maybe it could be a solution


@kwladyka I think it depends on what your data looks like, what you mean by doing something with it, and when you want it done. You could use different log levels and scrape the data later, or you might actually need to use message passing (core.async, manifold, even kafka).


sure, kafka etc. are good, but i want simple solution. I don’t know manifold. I will look on it.


@baritonehands What are the use-cases of that java services REPL?


@yonatanel Just like a clojure REPL, I wanted to provide a programmatic interface to poke at anything, kinda like a remote debugger


you can grab stuff out of the IoC container, query the DB, really anything


@baritonehands We run a bare Socket REPL in one of our production web app servers. It starts up on an unusual-but-known-to-us port and it is only accessible via the loopback address via telnet directly on that server -- the port is not open, even within the server's DMZ. It's proven extremely useful for debugging over the years (it used to be an nREPL server but we switched to the built-in Socket Server a while back).


Given that once you're physically on the server, you could start a REPL from the command line and run any code you wanted (from inside the JARs deployed there), having the process-embedded REPL accessible via telnet <port> doesn't seem any more problematic. What sort of objections have your colleagues raised?


privilege escalation?


@seancorfield Main objection was with devs running a REPL. It exposes more access than they currently have. SREs already have such access so it could be open to them. There was more concern over an accidental outage with good intentions than malicious use.


@bfabry Exactly, and much more terse explanation


Do devs have shell access to the production servers? I'm guessing not, in your case.


nor preproduction


The entire infrastructure was not designed with a REPL in mind. I think it would be possible to secure it, but may take some broad sweeping changes to the fundamental assumptions.


Right now I'm trying to find a way to get most of the features without the priviledge escalation


I agree that REPL access is pretty equivalent to full, unfettered shell access so if you don't have the latter, you shouldn't be given the former.


Given the power of the REPL, I'm not sure how you could ensure restricted functionality via a REPL-like connection. You'd easily have to write your own DSL and parser and interpreter I think...


...because restricting functionality would mean "not running Clojure code" -- it would have to be some DSL equivalent to a subset you wanted to allow.


clojurebot instance with a replacement for the irc ui?


I think a repl running under a user with only the privileges you want users to have would work?


we do have restricted access clojure sandboxes


clojurebot still allows you to run a lot of raw code tho'...


it does, yes


but if strangers on irc can use it…


@bfabry Not sure what you mean?


so like a separate clojure process that happens to be on the same machine?


yeah, and then regular unix permissions for access control


@noisesmith To be useful for debugging production issues you'd want SELECT access to any DBs, I'm sure, so the scope is pretty large. A separate Clojure process configured with only readonly DB credentials might be an option I guess.


@noisesmith I was thinking an actual REPL, pretty locked down in its own service, then a special interface for things it can do inside the other service


so you'd get a repl, data manipulation, utility functions


but only be allowed to do specific opt-in things in the other system


if so, clojail could be a place to start looking


but it sounds tricky to get right


You'd have to restrict which Java classes it could create, and which clojure namespaces were off-limits (,, etc).


@seancorfield For restricting Java access, a simple @REPL annotation could do


i would love to hear a follow up on how this turns out a month after it's put into place


I'd love to have a follow up


hopefully I don't get fired over this 😓


the utility of a repl is its generallity, once you start restricting what you can in it, it becomes less general


i echo @seancorfield 's recommendation for using a local socket REPL if possible


once you need to explicitlly allow everything you might as well just slap an api on it and access from a local repl


Alright, I'm off


I will play around with these ideas


thanks for the suggestions everyone


I love the clojurebot idea for this. Then these production investigations happen in public and knowledge gets shared with the team. Is this something you do on your team @noisesmith?


We have a socket repl, I thought of the bot as an example of solving, or nearly solving, the repl with restrictions issue.


When I need/want a sub-ns for spec, is there any particular reason not to just do:

(create-ns 'headmarc.rua.dkim)
(alias 'dkim 'headmarc.rua.dkim)
(create-ns 'headmarc.rua.spf)
(alias 'dkim 'headmarc.rua.spf)


lots of people do that, the only downside I'm aware of is lack of cursive support


why create the namespaces at all?


create-ns creates a namespace for holding code


spec namespaces are keyword namespaces


boot.user=> (alias 'foo '
java.lang.Exception: No namespace: found


the only place where code and keyword namespaces intersect is the '::' syntax, which is a tempting short cut, but a bad idea outside of playing at the repl


@hiredman why is it a bad idea? It’s pretty convenient when you have a bunch of large maps to type in.


lvh: because the names will change whenever you move the code around


the whole idea is stable names


(anyway yes that’s why; I understand how kw namespaces differ from regular namespaces)


yeah, type :dkim/foo then use your editors find and replace to change it to the real thing once you have typed all the maps


I would go further and recommend not using keyword namespaces that match a code namespace's name


spec names are like table.column names in a sql database, you don't name those after the code that reads or writes it


disagree with this. lots of keyword namespaces are good fits to be the same as a code namespace, though obviously not all. in the cases where it's not I still don't see a downside to having a real ns object for it and getting an alias


@lvh You can put the create call inline:

(alias 'dkim (create-ns 'headmarc.rua.dkim))
(alias 'spf (create-ns 'headmarc.rua.spf))
I think that's cleaner -- and will be less editing if/when alias is changed to auto-create the ns if it is missing (which is under discussion).


As for specs, I think there are cases where having them per @hiredman's suggestion is the right way and other cases where having them tied to actual code namespaces is the right way. It depends what they're a spec of and also what their intended scope is (public API, internal subsystem, opaque within a namespace etc).


@noisesmith how does the clojurebot solve the REPL with restrictions?


@baritonehands it provides a sandboxed repl where users have restricted permission - but others have brought up the complications that come into play if you need to access data or apis (things like the db for example) that the clojurebot isolation does not address


But you can add db/api access though? In the same way that plugins have been added for things like github links.


It seems it doesn't use clojail, how does the isolation work?


According to the Clojail readme, clojurebot does use it.


which clojurebot?


@seancorfield I thought only lazybot used clojail


Ah, maybe I misread the README then...




(isn't that how to invoke the bot here?)


I guess not...


the bot in slack, however it is invoked, is not that clojurebot


as far as I know, unless someone else is running and instance and wired it to slack


Anyways, @baritonehands You'd have to write some sort of pseudo-REPL that used Clojail to do the actual evaluation part -- so you couldn't just leverage the standard Socket Server REPL and Clojail together easily I don't think?


So it's /clj (some form)


The integration seems to be owned by @samflores and points to so I'm not sure which bot that is.


I'll take that elsewhere...


OK, so whatever bot is attached to /clj does use Clojail...


Having it respond "clojurebot" when it's not @hiredman "clojurebot" is confusing 🙂


have to namespace the nicks


I miss IRC...


(Know that I will repeat this every few months)


IRC is still there! @devn