Fork me on GitHub
#clojure-uk
<
2016-12-06
>
thomas08:12:11

aarrgghhh I can't start my repl anymore with out getting an error... and I have no idea what caused this. last week it worked fine 😡

thomas08:12:53

I deleted my .m2 repo, but the problem won't go away

tcoupland08:12:06

your project has some duplicate dependencies in it? perhaps the ordering of the jars has changed and your getting a different version first

Rachel Westmacott08:12:30

have you run lein deps since clearing .m2?

thomas08:12:56

I did a lein repl and that downloaded a lot of jars...

thomas09:12:39

hmmm I did add cucumber the other week... and might not have restarted the repl since... let me have a look

thomas09:12:42

thanks guys

thomas09:12:57

when doing a lein deps :tree I get the advice to loads of exclusions... is it adviced to do all of them? or just trial and error till you find something that works?

tcoupland09:12:34

depends how much time you have 🙂

dominicm09:12:22

@thomas I will add that lein can be a bit heavy handed with recommendations. Many of these things go in global exclusions, but lein will tell you to add them to every dependency.

dominicm09:12:32

But generally you should, yes.

thomas09:12:12

ok... I'll give it a try... thank you

glenjamin09:12:08

the other way to silence exclusions is to explicitly add the transitive dep at the top level

thomas09:12:29

this is a rather painful process... 😣

rickmoynihan09:12:23

glenjamin: +1 I find this approach describes the intent of the override/exclusion a lot better as it at least explicitly lets you pick and document the version that resolves the conflict.

mccraigmccraig10:12:54

@thomas i generally set :pedantic? :abort in my lein projects - it forces you to add all the exclusions required so that there is no more than one version of any given dependency in the tree

thomas10:12:51

well... when I do lein deps :tree I don't get any errors about more exclusions needed... but somehow it still getting errors somehow.

thomas10:12:39

I blame it on Rich Hickey... if I hadn't watched his talk last night about this problem I wouldn't even have known about this 😇

mccraigmccraig10:12:03

@thomas since you are getting NoClassDefFoundError you might have gone a bit far with your :exclusions

otfrom10:12:38

that would be my guess too. Or you excluded the wrong thing (conflict resolution suggestions aren't always correct)

mccraigmccraig10:12:41

i mostly find suggestions are exactly wrong... given two choices for an exclusion the suggestion is mostly the one i don't take

agile_geek10:12:12

I have a linux (specifically Ubuntu 16.04LTS) question and I know this is not a Linux channel but anyone mind if I ask?

otfrom10:12:41

I don't think we've been very good to sticking to our even vague topic

otfrom10:12:56

and the topic says "randomly chat"

agile_geek10:12:08

So hands up who has run ElasticSearch on Ubuntu?

agile_geek11:12:30

So I get a warning from ElasticSearch when I start it that the number of file descriptors is 4096 and needs to be 65536

agile_geek11:12:24

I followed at least 6 or 7 posts on how to up the number of file descriptors such that ulimit -Hn returns 65536

agile_geek11:12:56

All say pretty much the same

agile_geek11:12:41

set:

*              soft    nofile         65536
*              hard    nofile         65536
in /etc/security/limits.conf

agile_geek11:12:58

make sure the entry session required pam_limits.so is set in /etc/pam.d/common-session

agile_geek11:12:09

and log out and in

agile_geek11:12:53

I've tried every combination of this, rebooted at least 10 times today and tried that pam_limits setting in multiple files in /etc/pam.d

agile_geek11:12:59

However when I open a terminal and enter ulimit -Hn I get 4096 but if I then su - chris (my user) and try the ulimit cmd again I get the correct 65536

agile_geek11:12:30

so I have a work around but I keep forgetting to su in as me before starting elastic search

agile_geek11:12:00

it's frustrating and just wondered if anyone else had run Elastic Search on Ubuntu?

dominicm11:12:23

Are you using systemd?

dominicm11:12:17

Systemd is it's own pita grumble grumble systemd sucks grumble grumble

agile_geek11:12:41

I have no idea!

agile_geek11:12:08

I am just running the elasticsearch script from it's bin lib

agile_geek11:12:43

but the issue with ulimit -Hn is when I open a terminal

mccraigmccraig11:12:40

what's your ubuntu version @agile_geek ?

mccraigmccraig11:12:46

our production runs on ubuntu, but it's still on 14.04 lts

mccraigmccraig11:12:24

that's after the switch to systemd @agile_geek - best follow @dominicm's link

dominicm12:12:14

I wonder how limits work if ssh is a managed resource under systemd?

jasonbell12:12:06

Post ClojureX I realised I’m missing too much information with all the updates going on, so I’ve tried to curate things a little with a weekly update to myself. The Clojure Review https://www.getrevue.co/profile/jasonbelldata/archive/38086 feel free to signup and share as much as you want.

thomas12:12:55

@jasonbell done. thank you for this!

agile_geek13:12:05

@jasonbell nice work Jase! I think I would have added this too: https://www.youtube.com/watch?v=oyLBGkS5ICk&amp;list=PLZdCLR02grLofiMKo0bCeLHZC0_2rpqsz but there's just too much to reference when you have two awesome Clojure conferences on at the same time!

agile_geek13:12:07

I have had a lot of really good feedback about ClojureX talks from attendees of Conj who found them on SkillsMatter site so that's really 🆒

jasonbell13:12:17

It’s in the next one, no panic. I was putting issue 2 together last night.

otfrom13:12:49

too late. Already panicking. I'm in a right froth now.

jasonbell13:12:30

Another five days of froth it is, I will not be pressured by the huggerati.

jasonbell13:12:06

Wussy crying won’t help it along either.

otfrom13:12:32

Que es mas macho? Pineapple o 😭

jasonbell13:12:25

@mccraigmccraig you can if you wish but only if it’s on Soundcloud 🙂

otfrom13:12:38

there you go jasebell

jasonbell13:12:04

There’s always one isn’t there.

otfrom13:12:09

usually me

jasonbell13:12:16

And it’s usually bruce

thomas13:12:00

:hugging_face:

thomas13:12:09

or should it be

thomas13:12:16

:hugging_face: bruce :hugging_face:

jasonbell13:12:21

There you go.

jasonbell13:12:35

headings are borked 😞 oh well, for next time

otfrom13:12:07

wow, if that is all I have to do to get something shipped. 😉

jasonbell13:12:24

use it on the others and see how it goes

otfrom13:12:25

mostly I hug them. mostly.

thomas14:12:37

dang... this slack doesn't haev giphy integration...

thomas14:12:43

and that is probably a good thing.

dominicm14:12:52

@agile_geek Did the systemd thing help with your problem? I'd like to know so I have more to rant about systemd 😁

agile_geek14:12:28

@dominicm haven't had time to try. Wasted hours on it this morning. However I read the article and its about services bit my problem is simply the limit is wrong in the terminal from which I start elasticsearch from a script. Does systemd effect logins?

glenjamin14:12:02

does the elasticsearch start script switch user?

dominicm14:12:05

systemd is life. systemd overrides POSIX apis for fun.

dominicm14:12:34

^ that was about to be my next suggestion. Check the script, look for things like changing user, shell, HUP'ing is something else to look out for.

dominicm14:12:46

I should pay more attention, initial login has a reduced file limit.

dominicm14:12:02

Everything else is fine. Got it. Elasticsearch is a red herring 🙂

agile_geek14:12:25

It's not a script issue....If I start a terminal logged on a my user chris the file descriptors hard limit is 4096 and if I run the elasticsearch it complains it needs more file descriptors. If I start a terminal, su into my user again from the terminal the file descriptors limit is correctly set to 65536 as I specified in the /etc/security/limits.conf and if I run the elasticsearch script then it picks up the limit for my user

dominicm14:12:45

@agile_geek By "start a terminal" you mean gnome-terminal (is this desktop linux)

agile_geek14:12:00

Unity but yes

dominicm15:12:06

If you hadn't rebooted, I'd accuse you of not having logged out. This is strange. You're definitely into "something weird is happening" territory.

dominicm15:12:37

I can't help but feel like something really strange is going on. Like that ulimits are only applying to virtual connections and something about ConsoleKit checking. It's hard to tell with systems like Ubuntu, because they're so big now.

dominicm15:12:24

/etc/security/limits.d got anything in here?

agile_geek18:12:25

@dominicm only a casandra.conf file that has nothing to do with the problem. I did add the appropraite limits to the /etc/security/limits.conf as per that manpage

dominicm18:12:47

I'm declaring Ubuntu a casualty of war.

agile_geek18:12:16

I can workaround it

agile_geek18:12:34

If I su into my own user before starting ElasticSearch

agile_geek18:12:47

If I was using it a lot I would script it as a service

agile_geek18:12:45

Might just amend the elasticsearch script to set either the limits (if possible dynamically) or su into my user

agile_geek18:12:16

Wanted to solve it permanently in case other things need more open file handles

agile_geek18:12:31

Time for home anyway

paulspencerwilliams19:12:26

Quick favour if any Java heads are around? Am I being stupid whilst trying to reference PrettyTime NLP using

<!-- https://mvnrepository.com/artifact/org.ocpsoft.prettytime/prettytime-nlp -->
<dependency>
    <groupId>org.ocpsoft.prettytime</groupId>
    <artifactId>prettytime-nlp</artifactId>
    <version>4.0.1.Final</version>
</dependency>

paulspencerwilliams19:12:54

It's failing which might be related to our proxy Nexus running out of disk space

bigkahuna21:12:34

You should go to the .m2/repository and see if the artifact is actually downloaded. If there was a previous problem it can just leave a generated POM in the repository. When you try to use that dependency it won’t be resolved. What I usually do is delete the folder containing the artifact to force Maven to download again.

bigkahuna21:12:37

You can't force Maven to download release artifacts once it thinks they’re already downloaded. If it was a snapshot you could with a mvn clean compile -U which will force it to download the latest snapshots

bigkahuna21:12:09

Doesn’t seem to be available from mvnrepository

bigkahuna21:12:18

21:20 $ ll ~/.m2/repository/org/ocpsoft/prettytime/prettytime-nlp/4.0.1.Final/ total 48 drwxr-xr-x 2 bigkahuna staff 170 Dec 6 21:20 . drwxr-xr-x 3 bigkahuna staff 102 Dec 6 21:20 .. -rw-r--r-- 1 bigkahuna staff 173 Dec 6 21:20 _remote.repositories -rw-r--r-- 1 bigkahuna staff 2182 Dec 6 21:20 prettytime-nlp-4.0.1.Final.pom -rw-r--r-- 1 bigkahuna staff 40 Dec 6 21:20 prettytime-nlp-4.0.1.Final.pom.sha1

bigkahuna21:12:50

It generated POMs as expected ‘cos it can’t find the actual artifact

bigkahuna21:12:59

But 4.0.0-Final seems to be available

bigkahuna21:12:01

21:20 $ ll ~/.m2/repository/org/ocpsoft/prettytime/prettytime-nlp/4.0.0.Final/ total 5392 drwxr-xr-x 2 bigkahuna staff 374 Dec 6 21:22 . drwxr-xr-x 4 bigkahuna staff 136 Dec 6 21:22 .. -rw-r--r-- 1 bigkahuna staff 206 Dec 6 21:22 _maven.repositories -rw-r--r-- 1 bigkahuna staff 51520 Dec 6 21:22 prettytime-nlp-4.0.0.Final-javadoc.jar -rw-r--r-- 1 bigkahuna staff 40 Dec 6 21:22 prettytime-nlp-4.0.0.Final-javadoc.jar.sha1 -rw-r--r-- 1 bigkahuna staff 3922 Dec 6 21:22 prettytime-nlp-4.0.0.Final-sources.jar -rw-r--r-- 1 bigkahuna staff 40 Dec 6 21:22 prettytime-nlp-4.0.0.Final-sources.jar.sha1 -rw-r--r-- 1 bigkahuna staff 2643502 Dec 6 21:22 prettytime-nlp-4.0.0.Final.jar -rw-r--r-- 1 bigkahuna staff 40 Dec 6 21:22 prettytime-nlp-4.0.0.Final.jar.sha1 -rw-r--r-- 1 bigkahuna staff 2211 Dec 6 21:22 prettytime-nlp-4.0.0.Final.pom -rw-r--r-- 1 bigkahuna staff 40 Dec 6 21:22 prettytime-nlp-4.0.0.Final.pom.sha1

bigkahuna21:12:24

I know this stuff as I still work in the Java mines 😄