Hello All, has anyone faced/solved the problem of serializing Clojure classes using java.io.Serializable? What’s the idiomatic way of using serialVersionUID? (if there is one)

Don't laugh at me when I ask this: How can I get the name of an anonymous function function when it has a name?

For example: (fn cool [] (println "hi"))

how can I get the name "cool" from the function?

(.toString (fn cool [] (println "hi")))

@h0bbit what do you mean by clojure classes? records and deftypes? to be honest, the best answer is use something other than Serializable. Which is also the answer even if you're using pure java

@nick319 what do you need that for

hello guys

i have played with core.async a bit

https://gist.github.com/aibrahim/85af843ef94c6cda544e596f4ba8b50a

this a code which pass lists of numbers to each process linearly then each process try to remove multipliers of one number and pass result to other process

the code works fine for example till 10000 odds

when i tried to increase more for example 100000 it's broken

anybody can help in this?

thanks @gfredericks and @tanzoniteblack test.check’s shrink-loop looks like what I want 🙂

On database, Is it bad idea to enforce row level lock on a row, to avoid duplication in data?

what do you mean? that’s the job of the database, a unique constraint will handle it

Yes, i also thought that from database admin point of view, but php programmer wants to enforce in code

do NOT do that on code level

And some people wonder why i get annoyed at php devs that silly statements. hehe

just ask your PHP devs how well that would work when you have like, two processes (which happens pretty much instantly with PHP) doing that stuff concurrently?

cool. will do

We have 200 processes running concurrently

I’ll answer: it won’t work

since if it is in memory, then it works only for that single process

if it is on disk then one needs to take care of concurrency issues on I/O, meaning implementing transactions or something similar

and boom, you’ve done a database all by yourself, congratulations

postgres is acid complaint, so dont know why someone would consider it to be honest

most likely the person suggesting that hates databases in general and wants to keep everything at his hands and refuses to learn new stuff

always a fine combination

refuses to learn new stuff sounds like it, hates anything thats not mysql and php

see, almost a full match 🙂

and postgresql is actually much better than mysql when it comes to ensuring database integrity

Yes, i've learned that the hard way, hehe, been administering mysql for 5 years

So i replied to the dude suggestion with a link to postgresql unique constraint index

@niklas.collin so what could happen if someone were to enfoce row level locks, lead to issues with integrity?

s/enforce/enforce/g

you mean if you do it on code level, in this case with PHP?

yes

Just out of curiousity

oh god, so many things

I take it you have apache or nginx there spawning your PHP stuff, right?

no we use django and 5 web load balancers with nginx

he is not contributing to django codebase at the moment

right, doesn’t change things. Something somewhere is anyway spawning a PHP process per request

now, imagine two concurrent requests coming in, both of which try to do that row locking. If this row locking logic is kept in memory these two processes won’t see the locking defined by the other process since they are separate processes

if you’d have something sensible like Clojure then you could do that relatively safely - as long as you’d have only one server serving all the traffic

when you add another server then it’s the same problem all over again: different processes, different memoryspaces

So non-deterministic output?

it depends on completely what is being row locked and why

if row locking is done for writes then most likely this will fuck up your database

stuff that’s supposed to be unique is not

fuck up how, load average?

process A writes “foo”, process B tries to write “foo”, manages to do it since it doesn’t see the row locking stuff

Sorry i'm asking so many questions, but this guy will interogate and second guess what i know. So i need ammunition

in addition you’ll end up writing a lot of code which you could handle with single statement in your schema definitions for postgre

code which won’t work unless you create a mechanism to handle concurrency which is ACID compliant

that guy should write nginx too while he’s at it

Ok. thanks, I will just say no it wont work. Don't have energy to argue with know-it-alls

no, you can say that he can get it to work if he writes a fully fleged ACID compliant concurrency handling

good luck with that

especially with PHP 😄

roflol, but phalcan is a c implementation and is fast as hell

it’s not about speed, it’s about what you have at language level

PHP doesn’t have concurrency primitives

I know, you can write bad code in any language, most people write bad code in php, hehe

Is there a way to avoid a know-it-all outlandish ideas?

yes, get experience on multiple things, realize you know jack shit and get humble

I know, but i dont want to have to defend my position and knowledge all the time, its energy draining

I know my limitations, so i wont talk about stuff i dont know

been there done that. Changed jobs.

or to be more exact: client, not job

when I was a consultant

I really like this job, don't want to work for a corporate again

then deal with the ugly parts of it and be happy 🙂

Ok, good point

I think i'd be happy if i can get paid writing clojure code all day

but south africa is behind 😞

I’m actually writing scala these days, not exactly happy about that

Why, whats wrong with scala?

well, my main issue with it is the implicit and what it brings

Well, dont know any scala, so i can't say i understand 🙂

and also the ideology that “writing code is a bad thing, minimize the amount of code you need to write even if it makes your code unreadable”

which actually is part of why implicit exists in the language

but that “writability over readibility” is more of a design choice than a feature of the language

and still most stuff written in scala takes more code than equivalent clojure 😛

oh, can you migrate to clojure?

company is using clojure, team isn’t

trying to educate them 🙂

so we’ll see

the absurdity of it is that we are actually using datomic but not clojure…

odd

Heh, we have the opposite opinion in this team

Make it readable, even if it makes it a bit longer

yeah, I agree with that. But Scala peeps do not apparently

Makes it easier to onboard new team members, and easier to understand and bugfix when you come back to code you've not touched in mnths 😉

anyone tried kubernetes?

we use it at work

very expensive?

in what regard?

lets say i need 100 instances?

kubernetes is opensource, no licensing fees

so, it’s up to the price of your servers

So, how many instances can you have on decent server?

there is no clear answer to that since it totally depends on your containers

create distroless containers which contain just a hello world written in C: then the answer would be “millions” I think

ah, like alpine

alpine linux

no, distroless

oh

alpine linux is a distro

https://github.com/GoogleCloudPlatform/distroless

Very interesting

I think “millions” is a bit over the board, but “thousands” easily

but create a ubuntu container with weblogic behemoth inside and you can put your server to it’s knees pretty easily with just one container

I still need to learn kubernetes 🙂. Have to plan a project where a basic html page can handle 100k concurrent sessions

you do not need kubernetes for that: http://www.http-kit.org/600k-concurrent-connection-http-kit.html

i would not recommend http-kit

yeah, well most of the stuff there is about other stuff than http-kit

I guess you’d suggest aleph instead?

yes

Ok, just concertned about socket limit on servers and bottleneck on databases then

if it’s serving just html page you do not need a database

of course if you add databases into the mix it gets way more complicated

then kubernetes isn’t such a bad idea

Hmm, think they need databases, since it does api calls

ok

ok, so it’s not just a basic html page then 🙂

yeah, sorry, got ahead of myself

a side-effect of multi-tasking, hehe

but anyway, Kubernetes is a container orchestration platform like Docker Swarm or Apache Mesos (although that is actually more than just container platform)

so basically all it costs you is the servers where Kubernetes itself is running

rest is just your normal containers on top of VMs stuff, it’s just being managed by Kubernetes

I have yet another interop question, I'm pretty sure that the method .addExtensions exists for my imported org.openqa.selenium.chrome.ChromeOptions class.

(user/all-methods (new ChromeOptions))
(.addArguments .addEncodedExtensions .addExtensions .amendHashCode .asMap .getExperimentalOption .lambda$asMap$0 .setBinary .setExperimentalOption .setHeadless .setPageLoadStrategy .setUnhandledPromptBehaviour .toCapabilities)

(doto (new ChromeOptions)
        (.addExtensions (java.io.File. "chrome/my-extension.xpi")))
;; => CompilerException java.lang.IllegalArgumentException: No matching method found: addExtensions for class org.openqa.selenium.chrome.ChromeOptions

what feature are you using that makes aws lambda / google compute functions inadequate ?

@qqq asking me?

@niklas.collin: yes, I'm recently moved from EC2 + Elastic Bean Stalk to AWS Lambda + CloudFront + API Gateway + DynamoDb ... and am loving serverless.

well, we are talking about a huge microservice ecosystem here, hundreds of engineering teams

on top of kubernetes

we are doing whole bunch of stuff where AWS lambda wouldn’t be enough, especially performancewise

and long lived stuff which isn’t possible with that etc

I agree, lambda does not sound like the right use case for your project; good luck!

“project” 🙂

and FYI, it’s Zalando.

and all the other stuff under the hood which isn’t the fashion store

and lambda could definitely be used in some parts of the system

as it is now it’s not being used due to unpredictable costs at this scale

yeah; if you are running a huge shopping site, you should probably go the Amazon route and build Zalando Web Services

well, currently everything is on top of AWS

but I can see the company slowly building abstraction layer between AWS and Zalando stuff

like the Kubernetes change from EC2 instances

(doto (new ChromeOptions)
       (.addExtensions (list (java.io.File. "chrome/my-extension.xpi"))))

So, given a honey-sql statement as

["UPDATE my_table SET foo = ? where id = ?", "bar", 1]

is there a function in java.jdbc or honeysql which gives me

"UPDATE my_table SET foo = 'bar' where id = 1"

eg, a function from prepared-statement to string.

@niklas.collin how much of a learning curve is kubernetes

well it has some, but it depends on which side you are looking at: user of kubernetes as a dev or ops guy keeping it up

no experience on the latter

ok

@slipset probably not, given that most db drivers don't actually do any escaping in prepared statements - they pass the values directly to the db

@pesterhazy yeah, I figured as much after a while…

hi guys

nobody can help me

What are you looking for help with @abdullahibra?

hello guys [11:00] i have played with core.async a bit [11:00] https://gist.github.com/aibrahim/85af843ef94c6cda544e596f4ba8b50a [11:01] this a code which pass lists of numbers to each process linearly then each process try to remove multipliers of one number and pass result to other process [11:02] the code works fine for example till 10000 odds [11:02] when i tried to increase more for example 100000 it's broken [11:02] anybody can help in this?

How does it break?

@admay (def odds (rest (filter odd? (range 1000000))))

@abdullahibra, excuse me, I should be more clear. What error message do you get? Or do you not get any error message at all? 🙂

@admay https://gist.github.com/aibrahim/85af843ef94c6cda544e596f4ba8b50a

This doesn’t look like the entirety of the error output, what this everything that printed out?

how can i dump full trace error to file ?

what is the builtin for "pprint but out to string rather than stdout" ?

(with-out-str .... (pprint...))

@ghadi: thanks

@slipset That's an oft-requested feature for clojure.java.jdbc but as @pesterhazy notes, there's no easy way to get that from a JDBC driver under the hood, unfortunately. I guess my question would be: Why do you need/want this?

@seancorfield Yeah, I saw some discussion about it on the mailing list. My use case was the following. Given a csv file, I wanted to generate update statements. I wanted to have the actual statements so I could 1) have it reviewed by a teammate, and 2) add it to a migration script.

I ended up just running the honeysql statements through jdbc.

I realize that there are other ways of achieving this, but I had a nail and honeysql looked like a hammer at the time.

HoneySQL is a good way to compose query fragments, and producing vectors of ["sql statement" param1 param2 .. paramN] is a reasonable representation for review/debugging...?

True, and it’s what I ended up using for review. Doesn’t fit so great in a migration script though.

How can i read batch messages from channel?

@fantomofdoom xform w/ partition?

another option is a debouncer that pushes all values read after a timeout (time based batching rather than count based)

@potetm ok, but how i can set timer to not wait when channel fill to get me full partition?

@fantomofdoom that’s where you want a debounce loop and not a partition

I bet there’s a good library with a debounce in it out there, but there’s definitely gists showing how to do one in core.async

like this https://gist.github.com/scttnlsn/9744501

this isn’t exactly a standard debounce (since you want to collect all the messages over some timespan, and send all together), but the logic is very similar

ok, thanks

yeah- the modification to that would be to attach the new-val to the last-val, and make sure last-val starts as an empty coll

should be easy enough to make and unit test though

is it possible given a async/chan that returns maps say {:foo 2 :bar xs} where xs is a vector, to flatten transform the channel into one that yields elements of xs. (async/map< #(get :bar []) ch) gets me the vectors, but ideally I would like the flatten the whole channel (I know that map< is depreciated...

so if xs was a vec of ints (<!! ch) would yield an int

@beoliver a transducer should do that

https://clojuredocs.org/clojure.core/mapcat ?

(async/chan buf-size (mapcat :bar))

if I am given the chan (i.e I dont control its creation, should I create a pipe?)

^^^ Example of this being used

(let [c (async/chan 10 (mapcat :bar))]
                          (async/put! c {:foo 2 :bar [1 2 3]})
                          (async/close! c)
                          (async/<!! (async/into [] c)))

[1 2 3]

@beoliver yes, you can pipe from the channel you have with the maps onto a channel with that reducer, or just async/pipeline to accomplish the same thing without the transducer directly on the channel

👍

yeah pipe seems like the simplest solution there

or pipeline

is there a way to "perculate" the closing of the channel? can a/close? be made to close the channel that I have created a pipeline from?

it tends to go in the other direction, most things in the channel library have an option to close the destination channel if the source channel is closed

(the docstring for pipe describes exactly this)

hmm... I guess I will run some tests - just don't want dangling channels

pipe or pipeline will both propagate the close on the source channel. closing the destination channel doesn't make much sense as what do you do with the undelivered messages

flush the channel?

to where?

(loop [] ...)

to the ether

but...

seems bad man

the docs for pipe says it'll just stop consuming anyway

so does this look reasonable s/scroll-chan is out of my control. it just returns a channel.

(defn get-hits [response]
  (get-in response [:body :hits :hits] []))

(defn search-channel
  ([cli index type query-map] (search-channel cli index type query-map {}))
  ([cli index type query-map params]
   (let [ch (a/chan)]
     (a/thread
       (->> (s/scroll-chan cli {:url [index type "_search"]
                                :query-string (select-keys params [:search_type :request_cache])
                                :body (merge {:query query-map}
                                             (dissoc params :search_type :request_cache))
                                :exception-handler exception-handler})
            (a/pipeline 1 ch (mapcat get-hits))))
     ch)))

sorry for the extra crud

the a/thread seems strange there

as far as I am aware g/blocks are a bit iffy...

thread pool of 8

and not great with "long running" requests

I don't think you need either thread or go. the function scroll-chan returns a channel so is presumably asynchronous/fast. a/pipeline starts its own asynchronous operations so returns immediately. so you're creating a new thread that does 2 tiny operations then ends

shit... good point.

I do kinda think pipe would've made more sense than pipeline here.

(async/pipe
  (s/scroll-chan ...)
  (async/chan 1 (mapcat get-hits)))

yeah - makes more sense

pipe returns the to-chan btw. though it's not documented

scroll-chan has a ch optional arg that is the ouput chan. You can pass a ch with an xform for instance

I think all of spandex async functions allow to pass custom chans

And yes, scroll-chan is safe in go blocks it returns immediately, under the hood it s all async io

is that ok to use let instead of do for else block?

yes, let is a single expression, and it contains an implicit do block

Hi all, I'm preparing a Clojure talk at a local meetup (yay!) and re-watching this talk https://www.infoq.com/presentations/Are-We-There-Yet-Rich-Hickey in order to explain Clojure. At 52:34 Rich says "actors definitely do not" support "Point-in-time value perception", contrary to agents. I do not understand this statement, can someone try to explain it to me?

the way to get a value from an actor is by sending it a message and then it will send you message back. agents can be derefed

so it's the fact that "value perception" is deferred in the case of actors?

hola - does the latest leiningen force release/deployments to https? coworker complaining that it seems to be ( our nexus isn’t on https currently ).

you can't just read the sate of an actor right now, you have to effectively put your request in the actor's queue, and the actor will get to it when it gets to it

@arnaud_bos Unless you're specifically wanting to bring up agents, eg doing a talk about the Clojure STM, I'd leave them out of the talk.

the point-in-time is the key part there

Yep that sounds familiar @talios

@hiredman got it, thanks @slipset I just didn't want to follow through the talk and not understand this 🙂 I'll keep it simple this will be an introductory talk

As discussed in https://www.youtube.com/watch?v=9N7xM40fio0&list=PLZdCLR02grLqCR8EKuil8NrffZuTl0kwL&index=32 "nobody" ever uses agents 🙂

A talk very much worth watching though.

I think a more interesting thing, if you want to contrast actors with the clojure eco-system, is to contrast them to core.async.

Ah! Haven't seen that one yet, it's definitely on my list.

@talios https://github.com/technomancy/leiningen/commit/809c7d33ea5ef2ffefa1dc31ace839155955600d

Sure, I know for a fact that I'll have a few actors fans in the audience so...

http://www.dalnefre.com/wp/2010/06/actors-in-clojure-why-not/

@danielcompton kinda arse when its a 100% internal repo tho

I remember reading this one a few months back and not understanding every bits of the follow up discussion in comments with "Jeff Rose" is really interesting.

@arnaud_bos http://Clojure.org has a rather good overview on actors and "why not"

https://clojure.org/about/state#actors

And btw, http://dalnefre.com is a fantastic site, some really mind-blowing stuff on there. Not stuff I'd use at work everyday, but cool from a geek perspective

I've read that page on http://clojure.org a few times already and must admit I haven't fully made my mind up about this... 😅

I just don't use there concepts enough to get a good picture that is based on experience and not on theory and acquiescence bias

hi! I’m having problems with ring/compojure since upgrading leiningen. If I create a new app with > Lein new compojure testapp it creates the app like it should. But if I try to run it with > lein ring server I get this error “java.lang.RuntimeException: No reader function for tag object”

any idea how to fix this?

Is there a built-in "`split-around`" function? Sort of the opposite of split-at...I'm trying to say "keep the closest X values around Y in the given seq"

hey all, i ran into an issue with this foor loop

(for [gender     performer-genders
                               name       performer-names
                               instrument instrument-names
                               mood       moods]
  (do stuff))

basically, if any of those lists are empty, then the loop won't iterate over anything, which is expected behavior

the looping behavior im looking for is if one of the list are empty, instead of not looping over anything, do the iteration over the other 3 lists, just ignoring the empty list (binding the value in each iteration to nil or something)

does anyone know of a good way to accomplish that

oh wait

i guess i could just check to see if those lists are empty, and if so just bind the variables to [nil] seems like that should work

for isn't a loop, it's a list comprehension

if you want a loop, use loop

in particular where you say (do stuff) - if your intention is to do anything with a side effect rather than generating data, for is the wrong construct

nah that's just aggregating data within a reduce function

i guess loop would be better though since it'd have more explicit semantics then for