Fork me on GitHub
#clojure
<
2015-08-18
>
meow02:08:52

(apply sorted-set #{2 1 3}) vs (into (sorted-set) #{2 1 3}) any opinions or preferences and why?

meow02:08:13

I'm favoring the latter only because in nearby code I'm also doing an into (sorted-map) ... but the docs for sorted-set show the apply example

potetm02:08:42

@meow: Anything wrong with (sorted-set 2 1 3)?

meow03:08:26

@potetm: yes, because I'm taking an existing set and turning it into a sorted set for display purposes

meow03:08:03

so the actual code is (into (sorted-set) x)

meow03:08:47

so, nothing wrong with the code you show, just doesn't fit my context

meow03:08:22

I need to turn a set into a sorted set

meow03:08:19

I didn't state my original problem clearly enough.

potetm03:08:38

It doesn’t look like sorted sets are optimized for into (unlike, for example, hash maps). So, my guess would be there isn’t any (or much) efficiency advantage using one over the other.

potetm03:08:32

Could be wrong on that. I usually use into because I know it will use transients to speed up the process.

potetm03:08:57

But sorted sets don’t appear to implement IEditableCollection

meow03:08:37

@potetm: cool, thanks. I think I'll stick with into unless someone has a good reason to favor apply in this context.

aengelberg03:08:24

I just did some source code digging... sorted maps (which sorted sets are a thin layer on top of) don't seem to have a transient implementation or support optimized batch-addition of any kind.

meow03:08:09

@aengelberg: for what it's worth, my use-case for sorted maps and sorted sets has been when I want to display their values in the browser as sort of a debugging/developer aid. I've done that in my own app and I just added that feature to Devcards. So these are always bulk transformations from non-sorted to sorted.

onetom05:08:04

im playing with clojurescript on ec2 w 32 cores and it seems only ~8 core is used. is there any way to up it somehow? JVM_OPTIONS maybe? i know it might be specific to clojurescript itself but i hope there is some generic solution to this...

lowl4tency05:08:49

I guess it doesn’t relate to clojurescript.

onetom05:08:19

@lowl4tency: i found that page too but don't really see a conclusion on that page. the only lead might be the numactl but im a bit sceptical about that for some reason...

lowl4tency05:08:41

onetom: do you use linux?

onetom05:08:38

yup, im trying this on a default amazon linux

lowl4tency06:08:36

Share please your JVM options

onetom06:08:57

im using the defaults

onetom06:08:55

[[email protected] ~]$ java -version
openjdk version "1.8.0_51"
OpenJDK Runtime Environment (build 1.8.0_51-b16)
OpenJDK 64-Bit Server VM (build 25.51-b03, mixed mode)

lowl4tency06:08:19

and how do you recognize cpu using?

onetom06:08:43

ah, just using top

onetom06:08:35

the combined cpu usage was 20% and when i press 1 to see all cores independently, i can only see around 8 being utilized to 80-90% while using a 1s update frequency

onetom06:08:41

im also measuring the overall execution time of the cljs compilation which takes around 2mins on a 3.5 GHz Intel Core i7 iMac and it's around the same on 8core and 8+core machines too

onetom06:08:40

i tried a

[[email protected] ~]$ java -XX:+AggressiveOpts -XX:+UnlockDiagnosticVMOptions -XX:+UnlockExperimentalVMOptions -XX:+PrintFlagsFinal -version
and i see
bool UseNUMA                                   = false                               {product}
     bool UseNUMAInterleaving                       = false                               {product}
amongst the options...

wei08:08:47

Anyone know off the top of their head, the enlive selector syntax for getting the parent of a selected node?

grav08:08:10

@niwinz: when doing (require '[promesa.core :as p]), I get this error:

clojure.lang.ExceptionInfo: clojure.lang.Symbol cannot be cast to clojure.lang.Namespace at line 26 file:/home/grav/.m2/repository/funcool/cats/0.6.1/cats-0.6.1.jar!/cats/core.cljc {:tag :cljs/analysis-error, :file "file:/home/grav/.m2/repository/funcool/cats/0.6.1/cats-0.6.1.jar!/cats/core.cljc", :line 26, :column 1}

grav08:08:47

Using [funcool/promesa "0.3.0”]

grav08:08:12

woops, this should have been in #cljs

grav08:08:23

I’m in a figwheel repl

niwinz09:08:05

The promesa 0.3.0 depends on cats 0.5.0 but your error reports that you are using 0.6.1

niwinz09:08:33

cats 0.6.1 introduces some changes that are incompatible 100% with cats 0.5.0

grav09:08:14

oh, so it’s an implicit require from somewhere? i’m not much into maven

niwinz09:08:31

I will bump a new version of promesa with the cats dependency updated

niwinz09:08:38

Just wait 5 min.

niwinz09:08:48

(and continue speak on #C03S1L9DN

cfleming10:08:52

Is it possible to override an existing implementation of a multimethod?

cfleming10:08:32

i.e. if I require a namespace containing a defmethod, then in my namespace provide another with the same parameters?

slipset11:08:02

Bodil Stokke tweeted

slipset11:08:03

I'm not well versed in licenses and such, but I'd be very happy if someone could explain to me if this is correct, and if so, why...

onetom11:08:05

Sounds suspicious indeed... Maybe @richhickey is in the best position to answer this

dottedmag11:08:40

IANAL, but that is a correct statement.

dottedmag11:08:21

Well, the GPL code with exception to link against Clojure can be written, but one is unable to bring any GPL-licensed code into Clojure product.

Alex Miller (Clojure team)11:08:23

Standard not-a-lawyer caveats apply but I believe Mike Milinkovich's commentary is as good and accurate as you will find without consulting a lawyer

Alex Miller (Clojure team)11:08:57

There has been a lot of work on an update to EPL recently. They asked us for feedback and I provided some on Rich's behalf. I have not been following it closely since but Rich is entirely happy with EPL 1.0.

Alex Miller (Clojure team)11:08:31

It should not be surprising that GPL and EPL are at odds because their goals are different

slipset11:08:42

Maybe it would be nice with an article on http://clojure.org with some kind of official statement regarding this?

Alex Miller (Clojure team)11:08:55

@slipset: I do not know the implications of offering legal-ish advice

Alex Miller (Clojure team)11:08:35

That epl-discuss mailing list is a good place to ask questions about stuff btw

slipset11:08:58

Personally I really don't care about the licenses, but I would feel sad if people stopped using Clojure because of some problems with licenses, especially if those problems were non-existant.

slipset11:08:17

So I could imagine that it would be in the interest of Clojure that these issues were clarified, maybe even by someone who IAL.

Alex Miller (Clojure team)11:08:15

There is no way we are going to supply legal advice whether from a lawyer or not

Alex Miller (Clojure team)11:08:43

Most situations require knowing the context of your situation

Alex Miller (Clojure team)11:08:43

Clojure's choice of license was very intentional. From talking with him about it, Rich is well-versed in the intricacies involved here.

Alex Miller (Clojure team)11:08:49

As a user of Clojure and/or a creator of your own libraries or applications, it is your responsibility to read, understand, and comply with the licenses involved.

slipset12:08:49

I guess that is true, but that means that in the extreme case, each developer using clojure and creating her own libs or apps need to either be an expert in licensing or solicit advice from someone who is. I would have hoped that Someone(TM) could offer such advice once and for all for Clojure, but I see the problems of being the one who offers such advice. Especially if the one offering the advice resides in the states.

Alex Miller (Clojure team)12:08:58

Sorry, but that is the world we inhabit :)

niwinz12:08:59

I think that leaving all the responsability to the library developers is not a great approach, would be awesome that clojure maintainers / core team clarify all legal stuff

andrewhr12:08:23

as good as you can get without diving deep into this world

niwinz12:08:45

This is not about license choising, is about the implications of epl on the on the community

niwinz12:08:27

And posture like, "rich is happy with epl" may cause that a lot of great people leaves the clojure and its community just for maybe "wrong" decision of one...

niwinz12:08:43

This is not the first case and not the first cause!

dottedmag12:08:35

Well, being acquainted with licenses for a software developer is like being acquainted with personal finance for a grownup - many can chug along without it, but it might not be a good idea in a long run.

dottedmag12:08:35

There are similar situations in FOSS community (GPL vs. OpenSSL), so the solutions, workarounds and limitations are relatively well-known.

slipset12:08:54

@dottedmag: I would be inclined to argue that personal finance is somewhat easier to understand than why licenses are conflicting.

slipset12:08:36

@niwinz: I agree, and personally, I'd be interested in knowing why "Rich is happy with epl"

niwinz12:08:29

I think that we clojure core team should learn from errors of other comunities... I don't like that similar situations to nodejs/iojs will happen (unrelated to license problem, similar situation in terms of conflicts (decisions of one person/company...))

Alex Miller (Clojure team)13:08:01

Rich is happy with EPL == the license does the things he wants it to do.

Alex Miller (Clojure team)13:08:49

it's an approved OSI license. it's been vetted by IBM's lawyers. it derives its power from US copyright law which has vast legal precedent. the choice of law provision is NY (where Rich lives). it allows you to create derivative works, distribute them commercially, or use them privately. it contains patent grants. it requires source disclosure, license, and copyright attribution if distributed.

Alex Miller (Clojure team)13:08:24

the license was chosen because that set of attributes is what Rich wanted in the license.

Alex Miller (Clojure team)13:08:45

other software chooses licenses with other attributes. not all licenses are compatible.

Alex Miller (Clojure team)14:08:30

in other libraries (transit, transducers) we made different license choices - those are Apache 2 licensed for example

niwinz14:08:38

Yes, but the Rich opinion will not maters if the language community begins to reject its decisions and starts abandon clojure... I want to say that the community should be care also. Language without the community is nothing...

niwinz14:08:37

I think that Rich should hear those who use the language if he want that them continue using it.

Alex Miller (Clojure team)14:08:48

many people find EPL favorable to GPL because it supports a commercial ecosystem by not having the "viral" aspects of GPL. It's important to also be aware that there are philosophical and ideological beliefs at play around the GPL and FLOSS.

Alex Miller (Clojure team)14:08:38

in short, there are no plans to change the license of Clojure. it's not something open for discussion.

niwinz14:08:16

@alexmiller: it is ok, but would be awesome expose in some FAQ or something, all legal implications that the users of clojure shoult taka care about

niwinz14:08:28

To be more transparent and more community friendly

Alex Miller (Clojure team)14:08:40

we cannot and will not dispense legal advice. there is ample existing documentation about EPL.

niwinz14:08:47

instead of "the developer of third party library is the responsable to know that"

Alex Miller (Clojure team)14:08:57

you are responsible to know that

niwinz14:08:11

Yes, but the clojure team just can help

niwinz14:08:14

nothing more

niwinz14:08:28

we are not talking about responsabilities we talk about to be more friendly

damien14:08:27

@niwinz: I would say the majority of Clojure developers are perfectly capable of reading and understanding the implications of EPL. Not sure it matters if the core team offers any hand holding

roberto14:08:00

I don’t think this is clojure core team’s resp, nor the responsibility of any developer that has open sourced code. This is the responsibility of the license creators.

niwinz14:08:20

https://twitter.com/bodil/status/633566680879366144 is happens, And I think that clojure team should clarify it to its users... If the clojure team does not do it, the confusion can go further, and cause more people to be esceptic with the situation

roberto14:08:32

If I want to know something about the EPL, the first place I go to is the EPL’s website.

Joe R. Smith14:08:34

I don’t believe there is any precedent to having a library or language author explain the license they’re using

Joe R. Smith14:08:58

nor is there much motivation to be blamed for misunderstandings. simple_smile

roberto14:08:16

Bodil, if you follow her, is very ‘political’. Her views about the EPL are based on her personal politics, something not everyone will agree with.

roberto14:08:32

And she clarifies that in her tweet

niwinz14:08:26

I know and I understad it... but I think that clojure core team should clarify things for people that can not understand it, just that! But is my opinion obviously

niwinz14:08:30

nothing more 😉

roberto14:08:54

I mean, what she said applies to ALL EPL projects.

roberto14:08:34

so, I would naturally research more about the EPL instead of trying to find information on each EPL project.

roberto14:08:28

it is a license issue, not a project issue, and it is based on what one intends to do with that project and to some extent, some political (or lack) views.

sdegutis14:08:53

Does ring-jetty call your handler on a different thread per incoming request by default?

ordnungswidrig14:08:29

sdegutis: I would guess is a random thread from a fixed pool of threads.

ordnungswidrig14:08:54

sdegutis: why do you care? simple_smile

sdegutis14:08:00

The reason I ask is because I'm considering having a global mutable value that's shared between all requests.

sdegutis14:08:12

And I'm trying to figure out if that should be a var, ref, atom, or what.

sdegutis14:08:00

I know mutation is frowned upon in Clojure but the benefit seems to outweigh that: it would allow us to simplify our API immensely by removing a redundant variable from nearly every function

ordnungswidrig14:08:34

sdegutis: that is a good use case. I’d recommend using a ring middleware which assocs the value (or ref) into your request

sdegutis14:08:38

ordnungswidrig: that's currently what I'm doing, but then I have to pass that value to all my lower-level functions

sdegutis14:08:57

@ordnungswidrig: whereas if it was set to some global variable in the ring-middleware, I could omit it from all parameter lists

ordnungswidrig14:08:04

sdegutis: in this case you might want to bind a dynamic var.

sdegutis14:08:23

Ah, so you recommend ^:dynamic and binding?

ordnungswidrig14:08:33

sdegutis: yes, beware that you need to use bound-fn to make sure the binding is visible in all threads

ordnungswidrig14:08:12

Using a dynamic var makes it easy to set it during testing.

ianbishop14:08:03

Anyone ever written something akin to a single set var (that's not a local thread binding)

ianbishop14:08:27

i.e. having like an 'app-id' of a service, which is a global constant while the service is running

ianbishop14:08:50

but can't be determined by calling a function (i.e. can't do a fancy let over fn with a delay)

ianbishop14:08:00

it has to actually be set (exactly once)

ianbishop14:08:26

I mean, an atom is easy but more just curious if there's a way to do it that's not that

ianbishop14:08:44

and ideally not like a let over fn with a deftype and volatile-mutable or something silly, haha

bostonaholic15:08:08

@ianbishop: what about defonce like @alexmiller suggested?

ianbishop15:08:49

@bostonaholic: I need to be able to set it after read-time

ianbishop15:08:46

you can do some fancy stuff at get them evaluated at runtime by using a delay, but again that requires evaluating an expression

ianbishop15:08:48

rather than a set

ianbishop15:08:14

I just used an atom, but I was curious to know there was a secret clojure function as there sometimes are simple_smile

ordnungswidrig15:08:20

ianbishop: what should happen if you try to set it twice?

timgilbert15:08:42

Hey, just curious, why doesn’t (case) throw an error if you try to use it on something that isn’t a compile-time constant? Was recently bit by code resembling

(case (:status response)
  HttpStatus/OK (foo)
  HttpStatus/UNAUTHORIZED (bar))

Alex Miller (Clojure team)15:08:22

prob b/c those evaluate as numbers or something that are compile-time constants. would be ok by me to file a jira if you find something like this that doesn't work and doesn't throw

Alex Miller (Clojure team)15:08:56

it bugs me for interop that Java enums don't work properly (iirc) in case either. I think there might be a ticket for that one though.

Alex Miller (Clojure team)15:08:16

does anyone use (chan 0) in core.async with the intention of it being like (chan) ?

Joe R. Smith16:08:55

I’ve never thought of using (chan 0), does that work?

hlship16:08:42

@sdegutis: you are stumbling into the "easy" vs. "simple" realm. Not having to retrieve the value you need from the Ring request map is "easy" ... less duplicated code and so forth. But it is also an area of complection ... all of your functions will be less pure, since they will rely on this single bit of global state. Down the road, you may find things harder to test, bugs harder to find, and fixes harder to make, especially if you do this multiple times.

hlship16:08:25

I avoid globals like the plague unless there's some compelling reason ... and that's generally about Java interop, and needs to be carefully isolated.

hlship16:08:30

In Rook (https://github.com/AvisoNovate/rook), we have the ability to define argument resolvers for endpoint functions in terms of metadata and/or naming conventions. The upshot of which, is that an endpoint function may have an argument name db-conn, and via a naming convention, that's always the database connection obtained at system startup.

hlship16:08:35

So it's slightly less easy than a global: you have to setup the argument resolver, and know the convention name, and included it in your endpoint function, and pass it down the chain to other functions BUT there's no hidden state, you can test things easily, you can clearly see what "global state" is used by any endpoint.

sdegutis16:08:39

@hlship: Very true. That's why we've left the extra "db" parameter in nearly all our functions for the last 3 years. This is in a live website btw. We have no major globals for the reasons you mentioned.

sdegutis16:08:50

The only global we currently use is db-conn which during tests is set to an in-memory database, and when running live is set to a real database.

malcolmsparks16:08:01

I never liked using Ring as a mule for smuggling illicit refs to functions, lexical scope (as promoted by component) feels better, more general, not tied to Ring

sdegutis16:08:18

For now I'm leaning against storing the "most recent database value" in a global, in favor of just continuing to pass the database value around.

malcolmsparks16:08:05

Ring requests should represent an HTTP request, not a vehicle you can hijack for any old stuff you want to pass through

malcolmsparks16:08:07

We must come up with better patterns than this 'f*ck it that'll do' way of organizing our code bases

sdegutis16:08:22

@malcolmsparks: Even so, that's a process that takes years to figure out. And in the end, it's often still not significantly better (and sometimes it's actually worse) than what you started with. So there's a lot of risk involved and it needs much careful thought.

sdegutis16:08:35

Like, I'm making a change now to our codebase that I've been planning for about 18 months.

malcolmsparks16:08:53

@hlship - do you work for Aviso? (offtopic: I did some Java training in Killorglin years ago, lovely place)

sdegutis16:08:13

So far it's taken about 12 hours, so it's definitely not a lengthy task. But I put it off because it needed a lot of deliberation.

hlship16:08:39

I've been contracting there for about 2.5 years. That's winding down this month, but I'll continue to work on, use, and improve the open source libraries we've extracted from the code bases.

malcolmsparks16:08:53

@hlship - nice, good for you

hlship16:08:34

I originally did Tapestry 5 training for them. They've built a business around Clojure, which I believe they first learned about from my blog simple_smile

malcolmsparks16:08:17

@hlship - I remember that blog (and tapestry of course, it was used a lot at Deutsche Bank)

malcolmsparks16:08:43

@hlship are you based in Kerry then?

hlship16:08:03

@malcolmsparks: Although I can see where you are coming from in terms of using the Ring request map as a "mule", to me the HTTP contents are just the starting point, and I think anything is relatively fair game. Maybe I'll hit some issue that will swing my understanding.

malcolmsparks16:08:17

@hlship - it just 'feels' wrong to me, when there's a better alternative (passing dependencies via lexical scope) - but I can't properly justify it

hlship16:08:32

That being said, the Rook infrastructure does a clean job of supporting data extracted from a Component-based infrastructure. The app I was working on at Aviso was built around Component and Rook, and had essentially no global state stored in Vars; as you say, just lexically, or in atoms stored inside Components.

hlship16:08:20

In theory, we could run all of our tests in parallel against multiple Docker images (to store the databases) ... if Speclj supported that.

malcolmsparks16:08:26

@hlship yes, global state in vars is definitely a smell, smuggling stuff through the Ring request is definitely preferrable

malcolmsparks16:08:30

btw. we've been having some good experiences with running multiple databases (C*, pg, ES) via a single docker-compose, with joplin managing the state - it's nice

hlship16:08:03

Docker compose is one of the things on my list to check out, as some of my upcoming ideas will require multiple containers (i.e., a database container, a Redis container, a RabbitMQ container) even when developing and testing locally.

hlship16:08:42

We committed to using Liquibase for migrations; but that was when we needed to support H2 for fast/local testing, and PostgreSQL for production. Then along comes Docker, and the ability to support one set of migrations across multiple databases no longer makes sense.

hlship16:08:45

I'd like to see if Joplin supports my favorite feature from Liquibase though: the checksums that ensure you aren't trying to retroactively change history.

malcolmsparks17:08:00

@hlbase i'd have to check with @martintrojer

malcolmsparks17:08:26

even with docker, it's helpful to be able to migrate

hlship17:08:40

@malcolmsparks: agreed; migrations absolutely necessary, even with Docker. The feature we no longer needed was cross-database migrations that would adapt to H2 or PostgreSQL.

jonpither17:08:41

Joplin uses ragtime which uses the DB to store migrations that have been made, so it shouldn't re-apply past migrations. Though not sure what exactly the checksums do

hlship17:08:21

When Liquibase executes a migration, it stores the migration's unique id, and a datestamp, and a checksum derived from the migration data.

hlship17:08:42

When the migrations execute in the future, it compares the checksum for the migration in memory again st the checksum stored in the database.

jonpither17:08:55

@hlship: bumped into you back in 2008 in PDX at a neal ford talk on dynamic langs..

hlship17:08:13

If you change a migration after it has been executed against your database, Liquibase flags this as a severe error.

hlship17:08:22

And I find that useful.

hlship17:08:58

Migrations are supposed to be immutable. Otherwise, chaos and confusion.

martintrojer18:08:31

@hlship: like @jonpither says. joplin relies on the Migration idioms from ragtime. Checksum / hashing on migration history is not provided

hlship18:08:54

@jonpither: well, I live in PDX and all those talks kind of run together.

jonpither18:08:08

lovely place, I have to say

martintrojer18:08:35

I guess it depends how paranoid you want to be about your migrations.

martintrojer18:08:57

In some cases I can see the use for it, in other cases it would be an obstacle.

martintrojer18:08:52

I guess the idea here is that if you have high enough access to the DB to create / update tables, you need to understand what you are doing

martintrojer18:08:15

The final point about joplin is managing whole environments, that typically (in my experience) include more that just a SQL DB. I want to bring up ‘dev’ or ‘staging’ including Cassandra, ES, Zookeeper, RabbitMQ in a controlled manner.

upgradingdave19:08:41

Is there any datastructure/technique/library in clojure for treating a list of stuff that won’t all fit into memory as if it was all right in memory?

upgradingdave19:08:29

for example, if I have a s3 bucket with more items than can fit in memory, is there any way to treat the list of objects as a vector? something like: (nth objects 5001) would only load the 5001th object from the s3 bucket (or maybe load a batch of objects from s3 that includes the 5001th object)?

swizzard19:08:40

it might just be my OOP background talking, but my gut says deftype & extending the seq protocols you want to implement

upgradingdave19:08:47

thanks @swizzard, that makes sense.

Alex Miller (Clojure team)19:08:04

but you could implement Indexed

upgradingdave19:08:09

not sure how practical it would be but I wish I could treat everything as a list (database tables, s3 buckets, etc)

swizzard19:08:22

i was using ‘seq' loosely simple_smile

swizzard19:08:14

is there a good resource for clojure-internal protocols? i remember unsuccessfully looking a few months ago

swizzard20:08:24

good thing i apparently already bought it simple_smile

swizzard20:08:57

i remember that from some book or other

swizzard20:08:19

speaking vaguely of which: anyone have any good solutions for ‘utility functions’ in clojure? in e.g. python it’s easy enough to throw some garbage up on github and pip+git install it