Fork me on GitHub
#off-topic
<
2017-11-07
>
qqq06:11:15

for OCR purposes, is there a simple / standard algorithm for 'smoothig out jpeg artifacts'

borkdude07:11:55

@seancorfieldI just followed the weather forecasts that @annapawlicka posted on Twitter. Also the wildfires are caused by the heat I think? Half Santa Rosa burned down.

annapawlicka16:11:25

I don't live on the Peninsula. I live in East Bay, further inland, and it gets really hot here during summer. Whenever I want to cool off i just drive to Oakland, San Francisco, San Bruno etc. Wildfires were north of the bay but they could have happened anywhere in the bay area, probably. It doesn't rain in spring/summer/fall in most of the places here so the vegetation is super dry. Add dry air, wind, and electric company failing to trim dead tree branches and bushes.

seancorfield18:11:28

Yeah, once you cross the hills just east of me, the heat can be brutal. As for the wildfires, a decade of drought followed by a wet winter causing everything to sprout and grow followed by yet another warm, dry summer is a recipe for disaster!

seancorfield18:11:07

Re: PG&E trimming trees -- I called them out to look at one of my trees and because it is on my property, it's my responsibility, not theirs -- even tho' it's grown up all around the electrical cables! They only trim trees around the high-voltage lines now -- everything else is supposed to be county/personal responsibility. Like that will work... 😞

seancorfield18:11:30

(I paid a gardener to cut it all down)

borkdude08:11:21

Does anyone here ever use reducers?

pablore14:11:01

I do. What do you need?

borkdude15:11:09

Just wondering if people use it and what are their practical use cases

jsa-aerial15:11:20

All the time. I deal with loads of data (genomic data) that, if not always 'embarrassingly' parallel, is naturally parallel in most cases. From folding RNA, to alignments, to creating and matching distributions with RE, to ...

seancorfield18:11:28

I've never found a need for reducers (as in the library) but I'm finding I'm using transducers more and more in everyday code now, so lots of "reducing functions".

pablore20:11:52

You use it every time you want to get a single value from a collection of values. ie

(defn max-coll [vals] (reduce max vals)

seancorfield21:11:11

"reducers" not reduce -- or am I misunderstanding your response @U7V9HE682?

pablore14:11:49

yeah I was talking about reduce. But a reducer is any function you can pass to reduce

pablore14:11:32

You can also see a reducer function as a function that takes a state and some value/action and returns an updated state. Like in redux.

seancorfield16:11:27

@U7V9HE682 You need to be a bit more specific with terminology: "A reducer is the combination of a reducible collection (a collection that knows how to reduce itself) with a reducing function (the "recipe" for what needs to be done during the reduction)." -- https://clojure.org/reference/reducers

scknkkrer12:11:58

I want to improve my coding speed, any suggestion ?

souenzzo12:11:18

vim? macros?

schmee13:11:31

do you use touch typing? if not, start with that.

souenzzo13:11:35

Not sure if "type alot" will improve your "final" coding speed 😛

noisesmith13:11:36

I got fast at typing by playing MUDs - games where typing faster than other people is a major tactical advantage. But it's easy to get to a place with clojure in particular where you are typing faster than you are thinking, and there's no advantage to that.

noisesmith13:11:17

if you mean, how fast you can make correct, working code, the only way to speed that up is reading a lot of code, writing a lot of code, and fixing a lot of code

noisesmith13:11:20

there's no shortcut

danm14:11:51

Indeed. 80% of my coding time is thinking about how to solve the problem efficiently and in an easy-to-understand way. Only 20%, if not less, is actual typing

danm14:11:02

Sometimes more if I'm throwing ideas at the wall to see what sticks 😉

scknkkrer15:11:34

I don’t want any shortcut. I just need road-map and on the way learning. What do you guys suggest me for that ?

scknkkrer15:11:11

apart from Reading/Writing/Fixing code.

seancorfield18:11:32

@U3JJH3URY As one of the old guys here, all I can offer is "practice, practice, practice".

scknkkrer18:11:24

@seancorfield Thank you. I am practicing on exercism. Three days is not long, but I feel quite agile. But is there any new-generation technique about this. All suggestions are welcome.

seancorfield18:11:34

I don't think there are any shortcuts so it's really about setting aside time to practice as much as you can. Sorry.

mobileink19:11:40

master the tools. e.g. how quickly can you find the definition of a function you find in code, and where it is used? i can do it almost instantly (i use emacs and global; that's not the only way.) do you understand how your xompiler works (flags, search paths, etc)? no matter how great your thinking and design is, if you do not master the tools that help you actually do things you will suck as a programmer. because your work will be slow and error-prone.

tbaldridge14:11:27

@bcbradley @qqq as a counterpoint to last weeks discussion about Scala being a better experience, I have a friend who does Scala work in FL. He said their start-to-finish build times for their app are about 30min. With the toolchain taking >8GB of ram during compilation.

jcburley15:11:19

Do they have a good sense of how much of that is Scala overhead, versus the business-logic code (if it was implemented in other languages like Clojure or even C++)?

qqq16:11:56

@tbaldridge: I've heard horror stories about scala compile time. However, my current chocies, for certain tensors ops are: 1. clojure + type hints ==> slow anytime I make a mistake 2. write tensor code in java 3. write tensor code in scala

jcburley16:11:27

I have basically zero experience with most of that, so can't really comment, other than to sarcastically suggest they'd have lower build times if they used Python instead! 😉

tbaldridge16:11:51

I think it's a combination of several things. * Scala includes about 3 complete type systems * In order to stay sane a compiler of that complexity needs to be modular, that modularity normally carries a perf cost * Parts of scala is implemented/maintained by junior devs or students (not sure how much this plays into it)

tbaldridge16:11:12

But the end effect is that the compiler itself is very slow.

tbaldridge16:11:49

But I've also seen static typing make a impact on the size/quality of the code. In Clojure code, with maps there's very little "object conversion" going on. What I used to do C# code, about 1/2 of our app was conversion between DTO objects.

tbaldridge16:11:06

IIRC, full compiles of our app in C# took about a minute.

tbaldridge16:11:54

You get faster compiles by removing higher level concepts, but take that too far and you get GoLang...super fast...and a complete lack of metaprogramming

qqq20:11:23

@tbaldridge: it may be we're looking at different problems; in the problem I'm looking at, literally everything is an tensor of flloats (okay, maybe a few ints here/there for indexing)

tbaldridge14:11:35

So there's a price for all those fancy type features.

bcbradley14:11:50

i'm not a fan of types anyway

tbaldridge14:11:51

Also a key reason why Clojure is a 1.5 pass (or 2 pass, idk what to call it) compiler and not a nanopass

bcbradley14:11:34

honestly if i decide based purely on the tools, i'd choose clojure every day all day

bcbradley14:11:44

scala is good, but clojure is awesome

borkdude16:11:15

I’ve worked in Scala for 5 months but I came crawling back to Clojure. My feeling is you can make more momentum with Clojure, although this is absolutely subjective. Also I’ve only experienced it within one specific company.

borkdude16:11:38

There were many details people would discuss about that weren’t worth fuzzing about (imo) with regard to the business problems that needed to be solved. E.g. a discussion about returning Try[Unit] or Try[Boolean], or Boolean in a build script.

bcbradley16:11:29

i don't think scala would be a terrible language if everyone just dropped the type system and used Any for everything

bcbradley16:11:47

then do any data integrity verification manually with functions ala carte

bcbradley16:11:05

rather than relying on accurately specifying every type for every thing up front

borkdude16:11:37

It’s not a terrible language in theory, but it can be used in very different styles and attracts a mix of people that do not always form great teams together

borkdude16:11:18

A language like Haskell is better in that regard, I think

bcbradley16:11:24

i think the hope of scala was to allow object oriented professionals to work together with functional mindsetted people to allow cross pollination of different kinds of ideas

bcbradley16:11:49

it doesn't work out though because scala has so much syntax it makes it difficult for those two groups of people to talk to eachother

bcbradley16:11:53

same problem as c++

noisesmith16:11:23

things like this would be much easier if people just realized that Clojure is object oriented - it has all the good OO features

tbaldridge16:11:10

@borkdude I agree with you. I don't want to use Haskell, but I admire its purity and how it knows what it wants to be, and does it well.

tbaldridge16:11:49

Scala suffers quite a bit from trying to be all things to all people. You have to restrict your language or it will grow out-of-control. And end up like Scala (or C++)

chris16:11:39

if only we could post this conversation every time someone asks "Why doesn't clojure have feature x"

chris16:11:46

where's clojurebot when you need it

tbaldridge16:11:11

I used to beg for new features in Clojure. Today...I'm not actually sure that I want anything.

tbaldridge16:11:21

More spec stuff, but that can be done as a library

chris16:11:25

I just want a few new functions in core, but that's about it

chris16:11:55

core.async really opened my mind to the possibility of experimenting with things that in other languages would be built in, but in clojure is just a collection of functions and macros

borkdude16:11:09

I like that clojure is minimalistic

tbaldridge16:11:16

@borkdude I agree...if anything, I'd take more out of core.

tbaldridge16:11:26

.zip, .test, .reducers,

tbaldridge16:11:32

all that could be pulled out

borkdude16:11:38

@tbaldridge yes, e.g. letfn, I never use that construct

borkdude16:11:55

@tbaldridge yes, as libraries right

chris16:11:59

https://dev.clojure.org/jira/browse/CLJ-1959, this is what I'm referring to in particular

borkdude16:11:32

@chris I did upvote that one, because why not, I’ve needed it a couple of times

chris16:11:51

what I want more than new functions is an enhanced way to find the function I'm looking for

chris16:11:59

which is a tooling problem, but I'm not sure how to solve it

chris16:11:19

like, a few days ago I needed frequencies, but couldn't remember what it was called

borkdude16:11:32

In Haskell at least they’ve got Hoogle, but without types, it’s difficult. There is lein find-fn which is a cool way to find functions by example

hmaurer16:11:10

Hoogle is the best thing about haskell in my newbie opinion

chris16:11:33

there's so many functions already that it's hard to even know if something is in core

borkdude16:11:03

Haha yes. Recently I rediscovered complement, I always wrote comp not myself

bronsa16:11:44

letfn needs to be in core as it requires compiler support

hmaurer16:11:11

clojure.spec might make it easier to find functions if the functions in core get specs?

borkdude16:11:07

Yesterday for the first time ever I discovered the {foo :foo-key} destructuring, or properly understood that it allows you to rename the destructured key. Many features you don’t really need, but come in handy for writing more elegant code

borkdude16:11:23

I usually go for {:keys [:foo]}

dpsutton16:11:58

what does letfn require the compiler for that let [(fn [] ...]) can't accomplish?

bronsa16:11:45

mutual recursion

dpsutton16:11:08

ah ok. i see it registering locals. i wouldn't have thought of that

bronsa16:11:34

more generally letfn does forward declaration of all its locals

bronsa16:11:59

and the bindings are initialized as if they were being initialized in parallel rather than serial like let

dpsutton16:11:12

ah right. that's why there's the two for loops

bronsa16:11:05

yeah 2 loops for analysis and 3 4 for emission

bronsa16:11:13

letfn emission is quite more involved than let

borkdude16:11:03

I see! So what benefit do you have as a user from writing (let [f (fn [x] ...)] (f 1)) vs letfn?

bronsa16:11:28

in this case

bronsa16:11:36

but you do if you need mutually recursive local functions

bronsa16:11:45

that's literally its only use case

borkdude16:11:27

Ah now I get it

bronsa16:11:29

I guess some people like writing (letfn [(f [x] ..)] ..) better over let + fn even if they don't need letfn but eugh

val_waeselynck17:11:47

@bronsa you do need letfn for mutually recursive functions

bronsa17:11:02

yes i know, i've said that above

borkdude16:11:44

@bronsa > letfn needs to be in core as it requires compiler support what I actually meant was that I personally never needed this myself, but I can imagine other people do

borkdude17:11:08

Maybe it has to be there for some other features in core even?

bronsa17:11:22

nothing in core requires letfn

borkdude17:11:04

Something I’ve also rarely used is the STM stuff with dosync and refs

bronsa17:11:53

right, there's ton of stuff in clojure that's marginally useful

borkdude17:11:17

I can imagine that Rich has spent a lot of time on this 🙂

borkdude17:11:34

Maybe in Datomic this would be a needed construct

borkdude17:11:57

Can only speculate

bronsa17:11:01

it's more likely that some things seemed more useful 10 years ago than they actually turned out to be

jsa-aerial17:11:37

I don't think it is any less useful today, just that the overall number of use cases is fairly small. But when you have one of those use cases, things like the STM and letfn have off the chart value.

tbaldridge18:11:02

Yeah, I think both of those are cases where the problem is often reframed in a way that we don't need those features.

tbaldridge18:11:43

STM is often "replaced" by a single atom esp in systems where writes are rare. In more complex systems you often want the data store to be durable, and then Datomic or something like that replaces the need for STM

tbaldridge18:11:51

And letfn is a very function-centric macro, I think that's why I've never used it (except perhaps once, 6 years ago?). Most code I write is data-centric, so I would simply make two passes over the data, or run till fixedpoint.

tbaldridge18:11:28

Same thing with agents, I use the more than most clojure devs, but core.async has replaced 80% of my usage of agents

seancorfield18:11:04

One of the comments Alex made to me in a thread recently was that the tools.deps stuff, which grew out of the need for Clojure to be able to pull in clojure.spec and clojure.core.specs, will allow more of the "core" namespaces to be pulled out of the main Clojure JAR so they can then be evolved as separate libraries.

bcbradley01:11:29

If you are in the loop, consider trying to push for moving zippers out as well-- it could use a serious implementation rework (in terms of protocols for instance).

seancorfield02:11:59

@bcbradley Anyone -- including you -- is welcome to create JIRA issues for such things. I have no dog in that particular "fight" 🙂

seancorfield18:11:26

That was in response to me pushing on criteria for libraries being in Contrib (since one of the original -- and still -- stated goals for Contrib was a proving ground for code that could migrate into the core JAR). He said it's now more likely that code would go the other way. Which seems like a good thing.

bronsa18:11:15

i lied btw, mutual recursion can be implemented with combinators so technically doesn't need compiler support

bronsa18:11:25

((fn [x' y'] (let [x (x' y') y (y' x')] (x 6)))
 (fn x' [y'] (fn x [a] (let [y (y' x')] (if (zero? a) :even (y (dec a))))))
 (fn y' [x'] (fn y [a] (let [x (x' y')] (if (zero? a) :odd (x (dec a)))))))

bronsa18:11:30

is equivalent to

(letfn [(x [a] (if (zero? a) :even (y (dec a))))
        (y [a] (if (zero? a) :odd (x (dec a))))]
       (x 6))

bronsa19:11:29

needs labeled fns but that could be avoided with the y combinator too :)

ghadi21:11:41

Any jenkins experts in here?

dpsutton21:11:21

what's your problem? there was a change to checkout and tags that bit us recently if that's what you're dealing with

dpsutton21:11:37

not an expert but we dealt with that in particular

ghadi21:11:09

I'm trying to make a build trigger upon either push to master or open a PR. If I configure the hooks to be push events + open PR events, then I get duplicate builds when someone opens a PR. One build for when they push to their branch, another for when they open the PR.

khellang22:11:20

where are you pulling from?

khellang22:11:38

GitHub offers refs for PRs; +:refs/pull/*/merge

ghadi22:11:14

gitlab pushes to jenkins

khellang22:11:06

ah, I don't think there's a setting to filter refs in the gitlab integration

khellang22:11:26

the only way it'd work if all devs pushed to their own forks and PR'd back into upstream

khellang22:11:48

(which probably would be preferred anyway)

khellang22:11:28

hmm, looks like you can filter refs on the Jenkins side... https://github.com/jenkinsci/gitlab-plugin#branch-filtering