Fork me on GitHub
#boot
<
2015-06-21
>
juhoteperi11:06:56

Lol I've been wondering for some hours why I can't get the hyphenation to work

juhoteperi11:06:16

I'm using en-gb rules and but testing en-us 😄

martinklepsch11:06:53

@juhoteperi: do you still think task splitting is necessary btw?

juhoteperi11:06:35

@martinklepsch: Not sure. I'm not sure which phase would be the best for hyphnation or highlighting.

juhoteperi11:06:48

Highlighting should either find markdown code blocks or html code blocks.

juhoteperi11:06:11

hyphenation should change words in all except some html tags

juhoteperi11:06:40

I think it would be easiest to work with the html in hiccup representation

juhoteperi11:06:32

Another possibility might be a Pegdown plugin

juhoteperi11:06:07

Looks like Pegdown plugins could easily edit paragraphs and code blocks

martinklepsch11:06:31

My concern with splitting is mostly that we make assumptions about render functions, i.e. that they need to return hiccup. Imho it would be nice to be agnostic to whatever people want to use

mitchelkuijpers11:06:28

Can I help with perun http://prismic.io support? Have used it a lot from node but would really like a clojure solution @micha @podviaznikov

michal11:06:24

hi there. i’m just trying to run 2 boot repl sessions in the same project dir and seems it’s harder than I thought simple_smile first “boot repl” went ok and repl session has opened but second “boot repl” ends up with "Error: Exception thrown by the agent : java.lang.NullPointerException"

michal12:06:16

well.. it’s even worse. i’m not able to run boot outside of my project. NPE is thrown everytime as long as first session is alive.

martinklepsch12:06:18

@michal: what’s the output of boot -V

micha12:06:40

mitchelkuijpers: hi there!

micha12:06:09

mitchelkuijpers: what kind of applications are you making with prismic?

micha12:06:26

mitchelkuijpers: awesome, yes

micha12:06:55

how are you triggering rebuilds currently?

mitchelkuijpers12:06:57

Via prismic hooks

mitchelkuijpers12:06:12

Tot our build server (bamboo)

mitchelkuijpers12:06:03

Currently using baked.js which really sucks but has some good ideas

mitchelkuijpers12:06:43

And we deploy to S3

mitchelkuijpers12:06:01

Are you interested in prismic @micha ?

micha12:06:15

mitchelkuijpers: yes, very much so

mitchelkuijpers12:06:32

Will you be at Euroclojure?

micha12:06:33

mitchelkuijpers: i have a project coming up soon that will be using it

micha12:06:44

i wish i could, but no 😕

micha12:06:08

i plan to be in finland in sept though, for clojuTRE!

mitchelkuijpers12:06:33

Hmm might have to go there

micha12:06:46

mitchelkuijpers: where are you located, if you don't mind my asking simple_smile

micha12:06:41

glorious, i bet the weather is awesome now

mitchelkuijpers12:06:55

A lot of rain haha

micha12:06:38

haha i was imagining bicycling with picnic baskets in fields of tulips

micha12:06:56

i'm in durham north carolina

mitchelkuijpers12:06:26

Well I might go to clojutre looks fun!

micha12:06:52

just woke up, watching deer and creatures walking around in my yard

mitchelkuijpers12:06:22

Anyhow if you need help with prismic don't hesitate to ask been using it for 2 years now

micha12:06:01

great, in about a month or so we should be starting the prismic integration

micha12:06:28

i'm not sure if we want to go with static site generator or single page app consuming prismic data in the client yet

micha12:06:15

possibly a combination of the two, with front page content handled dynamically in the client and archives/permalinks via static site

mitchelkuijpers12:06:54

Sounds like a fun project

mitchelkuijpers12:06:16

A combination is pretty easy todo

micha12:06:36

prismic seems pretty great so far

mitchelkuijpers12:06:03

Yes never failed us

mitchelkuijpers12:06:24

Not perfect but better then the alternatives

micha12:06:34

anyway whatever we end up making will be open source (eg perun modules, etc)

mitchelkuijpers12:06:28

Cool, would love to help out want to do more open source clojure

mitchelkuijpers12:06:58

I am the only clojure developer @ work so don't get a lot of feedback

micha12:06:03

awesome, we will collaborate!

michal13:06:49

@martinklepsch: #https://github.com/boot-clj/boot #Sun Jun 21 15:04:15 CEST 2015 BOOT_CLOJURE_VERSION=1.7.0-RC2 BOOT_VERSION=2.1.2 #App version: 2.0.0

micha13:06:08

michal: can you paste the stack trace please?

micha13:06:41

michal: you can use boot -vv ... to get a full trace with no colors etc.

michal13:06:00

@micha: the problem is, there is no stacktrace

michal13:06:01

~ % boot repl -vv Error: Exception thrown by the agent : java.lang.NullPointerException

michal13:06:31

do you observe the same or it’s just only my problem?

micha13:06:35

whoa, never seen that one before

micha13:06:45

which OS are you on?

michal13:06:55

run one repl session somewhere and try just to run boot somewhere else

michal13:06:11

Ubuntu 10.04.4 LTS

micha13:06:24

i have like 5 repls going in different tmux sessions all the time

juhoteperi13:06:29

user=> (clojure.string/replace (hyphenate-word fi/rules "lentokone") +hyphen+ \-)
"len-to-ko-ne"

micha13:06:33

it's hard to track them down sometimes simple_smile

juhoteperi13:06:45

eh, soft hyphens aren't displayed here

michal13:06:27

maybe I should upgrade java or something...

micha13:06:40

which version of java?

micha13:06:50

i mean i test with java 7 and 8

micha13:06:16

java 6 and lower isn't supported by boot, because boot uses the java 7 nio stuff

michal13:06:06

~ % java -version java version "1.8.0_05" Java(TM) SE Runtime Environment (build 1.8.0_05-b13) Java HotSpot(TM) 64-Bit Server VM (build 25.5-b02, mixed mode)

micha13:06:42

do you have a ~/.boot/profile.boot file?

micha13:06:44

deraen: dude that's sweet!

michal13:06:28

ahhh I know what happened. sorry for taking your time, I’ve added -Dcom.sun.management.jmxremote to BOOT_JVM_OPTIONS to have jmx enabled

micha13:06:40

michal: what were you using jmx for?

micha13:06:05

we might want to add support to boot for this

michal13:06:12

for remote tracking cpu/memory usage via visualvm/jconsole

michal13:06:29

that would be really awsome simple_smile

micha13:06:31

seems like a thing boot could handle for you in a nice way

michal13:06:35

I guess it would be still based on jmx?

micha13:06:24

i'm not very familiar with jmx, i'm embarrassed to say

micha13:06:34

i've used visualvm and stuff

michal13:06:48

well, it’s really useful thing in java world simple_smile

micha13:06:15

what do you think the problem you ran into was?

micha13:06:39

lol it looks like jmx is kind of a thing

micha13:06:57

it's preposterous that i'm not using it

michal13:06:10

I think, the problem was the same port I was trying to use. full BOOT_JVM_OPTIONS look like this:

michal13:06:11

-Dcom.sun.management.jmxremote -Dcom.sun.management.jmxremote.port=7091 -Dcom.sun.management.jmxremote.rmi.port=7091 -Dcom.sun.management.jmxremote.authenticate=false -Dcom.sun.management.jmxremote.ssl=false

michal13:06:37

and most likely java is not able to establish 2 jmx agents at the same port

micha13:06:46

ah that makes sense; it couldn't start the second server on the same port

micha13:06:31

port 22 is the only one you need maybe

micha13:06:51

since jconsole can attach to running processes without the system property set

micha13:06:09

ssh --> jconsole?

michal13:06:15

actually it’s more complicated if you have machine behind firewall. jmx uses 2 ports to communicate

micha13:06:41

oh bummer

michal13:06:13

as far as I know these 2 ports might be exactly the same, but still you have to provide them explicite in jmxremote.port and jmxremote.rmi.port

micha13:06:31

maybe boot processes can cooperate somehow?

michal13:06:21

sure. boot can start up jmx and provide additional beans, statistics and things like this. it’s really useful for regular java based servers, so maybe boot could also utilize that

michal13:06:37

jmx gives almost endless posibilites to extend applications but usually it’s used for gathering statistics

michal13:06:45

dropwizard’s metrics (https://dropwizard.github.io/metrics/3.1.0/) nicely integrate with jmx

micha13:06:26

michal: this sounds great, if you would like to sort of sketch out some initial requirements i will write the code simple_smile

micha13:06:00

i'm not experienced with this but i'm keen to learn

michal13:06:55

sure! i’m just creating clojure app with jmx extensions in mind, so I will try to gather and describe some of my ideas simple_smile

micha13:06:13

we use datadog extensively at my job, i'm imagining doing stats for applications via jmx maybe

micha13:06:30

datadog agent has a jmx plugin i think

micha13:06:46

which probably could discover the metrics and whatnot

micha13:06:11

so the metrics in my applications would be agent-agnostic kind of

micha13:06:46

like if we want to connect a different collector to the jmx agent we could and then we'd have metrics there too

micha13:06:19

i guess this is all basic stuff in the java ee community

micha13:06:12

with clojure i imagine we could write some macros and functions and whatnot to present an api applications can use to send metrics that's as easy to use as datadog

micha13:06:25

but via jmx

michal13:06:02

yeah, I used that one. it works simple_smile

micha13:06:53

like if i wanted to monitor latency of dynamodb update requests, for example, i could make an mbean for that?

micha13:06:59

in my appliaction, i mean

micha13:06:20

or is it strictly for JVM related metrics?

michal14:06:29

jvm related metrics (cpu/memory/thread and so on) you have for free. any other metrics you have to provide to jmx by yourself. I’m doing it via https://github.com/sjl/metrics-clojure

michal14:06:03

jmx tools like jconsole or visualvm give you even realtime graphs to observe how these metrics change

micha14:06:14

dude this is great

michal14:06:06

but what’s most interesting you may integrate any other tool that is able to connect to jmx and consume your metrics in your special way

micha14:06:28

that's what i'm interested in, too

michal14:06:28

eg. you may send them to riemann and show on grafana simple_smile

micha14:06:44

it's like a metrics manifold

micha14:06:55

you can tee off multiple collectors

micha14:06:15

man this is exactly what i have been needing but didn't know it

micha14:06:44

it could be a general purpose discovery thing too i bet

michal14:06:44

glad I had a problem with jmx and mentioned it today 😉

micha14:06:13

like have an mbean exposed via jmx that has application repl port

micha14:06:21

or even jmx command to start a repl

micha14:06:28

in different contexts in the application

micha14:06:33

or for pods!

micha14:06:51

boot could expose jmx hooks to start a repl in a pod!

micha14:06:23

like you could see a list of all pods, and send a command to a pod to start a repl server you can connect to

micha14:06:47

to debug issues that are happening in some pod that's not accessible directly from the build.boot context

michal14:06:38

hah! sounds a bit like inception (pod’s repl in repl) simple_smile

micha14:06:55

everything good in computers is recursive

michal14:06:55

anyway, the basic thing that would help a lot would be just a jmx plugin which would expose jmx agent on given port (possibly with user/pass) and attach some mbeans

micha14:06:46

how does jmx interact with GC?

micha14:06:14

like if i register mbeans do i need to worry about references being held somewhere that will prevent things from being collected?

michal14:06:08

garbage collector? you may enforce garbage collecting via jmx, but as you probably know it doesn’t really mean that GC will release mem immediately. it works more like a “suggest”.

micha14:06:44

i'm more concerned with having refenreces to global things (jmx agent etc)

micha14:06:03

if those references will prevent objects from being garbage collected

michal14:06:08

I guess you would have to take care of references by yourself

michal14:06:21

there is no magic behind

micha14:06:50

that metrics library, A+

michal14:06:59

it’s the most popular and used literally everywhere (or at least in my company 😉

micha14:06:35

and it has a datadog extension

michal14:06:55

I didn’t even know it simple_smile

micha14:06:11

this is especially cool because with this we have a way to compare datadog against something

micha14:06:23

like we can connect and see graphs via dropwizard

micha14:06:34

sometimes datadog has hiccups and alarms go off

micha14:06:39

this would solve that problem

micha14:06:37

which other java framework type tools do you use?

michal14:06:17

we also use riemann with influxdb to store metrics and feed grafana

michal14:06:15

influxdb was quite new for me, I didn’t know there is a database for metrics only

michal14:06:34

maybe that’s also something you could try

jeluard14:06:41

Is there a way to have new snapshot dependencies picked up during a boot run?

michal15:06:20

remove old ones from local .m2 repo :)

jeluard15:06:22

Will that make a running boot automatically pick up the new one? Say in the next watch triggered cycle.

jeluard15:06:48

I have 2 running boot, one with dependencies on the other. I'd like to have changes in the dependent one being picked up without having to restart my boot process.

micha15:06:25

jeluard: yes!

micha15:06:31

boot checkout --help

micha15:06:39

i think that does what you need

micha15:06:41

i use it for example when working on the hoplon compiler, where i'm working with an example application for instance, but i want modifications to hoplon compiler to be part of the watcher pipeline

micha15:06:19

in the hoplon project i do boot watch build-jar

micha15:06:00

that runs an incremental build that builds hoplon and installs the hoplon jar in local .m2 whenever anyhting changes

micha15:06:20

then in the example application where i'm debugging and exploring i do

micha15:06:49

boot watch checkout -d tailrecursion/hoplon:6.0.0-alpha2 dev

micha15:06:05

where the dev task builds my local project that depends on hoplon

jeluard15:06:01

Awesome! That really is a killer feature.

micha15:06:55

it's more powerful than :checkout-deps in lein because everything synchronizes around the jars

micha15:06:17

so your checkout build could be doing a lot of things that the secondary project doesn't need to know about

micha15:06:31

it just consumes the jar

jeluard15:06:13

Right this is much more user friendly.

jeluard15:06:08

Next level would be to enable this for all dependencies (like checkout :all ?) and even better to resolve changes to the buid.boot simple_smile

micha15:06:26

how do you mean?

micha15:06:51

oh like annotate :dependencies in the env?

micha15:06:24

wow this is an interesting concept

micha15:06:27

i like it

jeluard15:06:29

I mean that I don't have to specify the checkout dependencies but it just resolves my build dependencies.

jeluard15:06:06

Does checkout injects the dependencies? It's not related to my build dependencies?

micha15:06:31

checkout actually conflicts with your build dependencies

micha15:06:44

you should remove the checkout dep from your :dependencies when you're using that

micha15:06:56

the checkout task watches the .m2 repo

micha15:06:05

and unpacks the jar into the fileset

micha15:06:30

it does look at the jar's pom.xml so it can resolve transitive dependencies

micha15:06:47

well it uses aether to resolve the checkout dep's transitive deps

micha15:06:53

and adds them to your project deps

micha15:06:08

so it could have unexpected side effects in some cases probably

micha15:06:18

but it mostly works as you'd expect it to

jeluard15:06:11

Would it make sense to have build dependencies resolved per 'run'? Not sure if this is technically feasible.

micha15:06:34

the problem is that there is no way to remove jars from classloaders

jeluard15:06:39

yes this is already great. The downside is that you can't really have a generic build file.

micha15:06:03

we'd have to make a fresh pod for each run

micha15:06:14

which is doable but has other ramifications

micha15:06:24

like how to keep state between runs

micha15:06:35

ultimately i think this is the direction i'd like to go

micha15:06:51

with each run being almost a pure function that takes fileset->fileset

micha15:06:08

but there are things like side effects that make this a little more complicated

micha15:06:22

(starting a web server for instance, things like that)

micha15:06:41

what i imagine is maybe a sort of session store

micha15:06:54

kind of like the http server model

micha15:06:49

that would solve all kinds of annoying problems, for example it would make reloading of clojure code obsolete really

micha16:06:14

that's a place where you get annoyingly hard to debug problems

micha16:06:32

like your tests are passing for an hour in the repl

micha16:06:38

you commit your changes

micha16:06:00

and then you realize that you've been writing code for the past hour using some stale state in the repl

micha16:06:07

and none of your stuff works anymoer

jeluard16:06:28

Sounds very promising! Great to see you have plans to push boot even more.

jeluard16:06:04

Also I was wondering if boot had the necessary hooks to have some dashboard similar to https://github.com/oakmac/cuttle Expect it could be a generic tool for any build.boot.

micha16:06:36

this jmx stuff seems like a perfect fit here

micha16:06:06

i imagine maybe if we add jmx support to boot and boot tasks participate, you can use a number of already existing jmx frontends to visualize and interact with the running boot process

juhoteperi16:06:58

@micha: https://www.dropbox.com/s/bolelkz613aocs7/906a6334-04a0-4153-b3c1-ca0da5e8b022.png?dl=0 Hyphenation is sort of working (for some reason some words are hyphenated incorrectly :D) but CSS columns are horribly broken

jeluard16:06:09

Right. jmx is nice but pretty constrained and limited to the Java world. A dumb HTTP server would allow to interact with something as simple as curl.

micha16:06:47

jeluard: i bet there are lots of awesome http-->jmx things out there already that we could use

micha16:06:40

deraen: that looks great!

micha16:06:59

what do you mean by css columns being broken?

micha16:06:22

i guess "widows" and "orphans"

juhoteperi16:06:36

@micha: Padding and margin from previous column continues into second column, so the column contents start in random positions

jeluard16:06:54

Hmm actually it's the other way around.

juhoteperi16:06:17

@martinklepsch: @podviaznikov: Very rought version of hyphenation: https://github.com/hashobject/perun/compare/hyphenation

micha16:06:01

dude good job on clj-hyphenate

juhoteperi16:06:08

It is currently tied to clj representation of markdown AST

juhoteperi16:06:48

@micha: it still makes some errors, "let-t-ers" vs. "let-ters", "hy-phen-a-tion" vs "hy-phen-ation"

micha16:06:49

seems like some exclusion rules are missing?

micha16:06:24

i started reading the paper yesterday, it's interesting

juhoteperi16:06:39

patterns from hyphenator.js don't include exclusion rules

micha16:06:01

does it use the liang algorithm?

micha16:06:12

but only one pass

juhoteperi16:06:25

Exclusions can also be represented as regular patterns

juhoteperi16:06:54

e.g. ta-ble would be 6t6a7b6l6e6 or something 😄

micha16:06:36

in the paper they describe applying patterns, then exclusion rules that override the patterns, then inclusion rules that override the exclusions, then exclusions ... 5 layers deep i believe

micha16:06:32

it's a pretty sweet algorithm

juhoteperi16:06:43

Yeah I glanced through the paper quicky

podviaznikov18:06:57

@juhoteperi: clj-hyphenate looks useful. I wasn’t aware of http://mnater.github.io/Hyphenator/ - I like how text is more readable that way (and takes more horizontal space). I’m going to try it

danielsz22:06:10

Did you notice that Slack was written in atom-shell? Which is javascript-based. Which makes it compatible with Clojurescript. But who has written/will write a boot-atom-shell task to build a project?

micha22:06:40

danielsz: have you done anything with atom shell?

micha22:06:47

i don't know anything about it really

micha22:06:09

also this build tool is really sweet: https://github.com/apenwarr/redo

danielsz22:06:14

Nope. It was a bit tongue-in-cheek.

danielsz22:06:40

About redo, this is legendary stuff. Daniel J. Bernstein laid out his vision for this, then never wrote it.

micha22:06:10

it's fantastic

danielsz22:06:28

You’ve played with it?

micha22:06:35

no, just read the design doc

micha22:06:08

it's an outstanding example of that

danielsz22:06:44

Has it influenced in any way or shape boot itself?

micha22:06:10

i just discovered it 😕

danielsz22:06:56

Off-topic: I’ve spent an epic few days rethinking my email workflow, and I switched from a regular Mail.app/Gmail setup to a mbsync + msmtp + mu4e + emacs setup. It still needs to pass the test of time, but I already enjoy the benefit of a 100% portable configuration across the board (mac and linux, server and desktop). Config is under revision control (except credentials of course), exactly like dot files. It feels amazing.

micha22:06:48

interesting

micha22:06:02

i haven't used a mail application since like 2004

danielsz22:06:16

what are you using?

micha22:06:27

i always ended up with gigs of corrupted garbage after a while

micha22:06:34

i just use gmail via the web

juhoteperi22:06:08

I think gmail is more portable than alternatives. It works on Android and Pebble :D

danielsz22:06:19

exactly! That’s exactly what is happening. gigs of corrupted garbage.

micha22:06:44

yeah i realized that i don't really care about old emails

micha22:06:01

the important thing is being able to receive a timely message

micha22:06:25

and the continuity, like my email address has been the same for like 15 years now

danielsz22:06:47

deraen: email on the pebble? lol

danielsz22:06:28

mbsync is an amazing program.

danielsz22:06:33

msmtp also, actually.

juhoteperi22:06:03

@danielsz: Well, the notifications are great and voice reply seems to work surprisingly well. (Obviously nothing prevents using Gmail on Android when using mailclient on desktop)

danielsz22:06:20

deraen: cool.

juhoteperi23:06:41

@micha: Should be a lot more readable now: https://github.com/Deraen/clj-hyphenate/blob/master/src/clj_hyphenate/core.clj still does't work correctly though. I guess I need to take a look at pattern parsing & trie building.

micha23:06:18

deraen: looks good so far