Fork me on GitHub
#off-topic
<
2018-01-16
>
qqq00:01:46

Anyone know of a slack / irc / ... where the CUDA experts hang out at ?

qqq03:01:36

Has any company ever gotten the following to work for recruiting / hiring ? 1. Here is a list of the top 20 libraries we use at FooCorp. 2. If you want to interview at FooCorp, pick one of the libraries, resolve a few bugs / add a few features, get 100 lines of code added to your name.

seancorfield03:01:04

@qqq Requiring public open source contribution will exclude a lot of candidates so, no, I wouldn't expect that to work.

qqq04:01:02

Hmm, maybe the simplest way to get contract work done is to just hand open source devs contracts of the form "We need lib XYZ to add feature FooBar, you already know XYZ, will you do it for $10k? You don't have to merge it into trunk, and you are free to open source the work as you wish."

qqq08:01:10

blas supports saxpy which ys ax+y where x,y are vectors and 'a' is a scalar

qqq08:01:20

any idea why the primitive isn't ax+by where a,b are scalars ?

andy.fingerhut10:01:51

I do not know why, but have a guess - for most purposes in linear algebra, any constant factor (except 0) of a row or column is 'equivalent' to any other, for purposes of Gaussian elimination, and probably most others. ax+by = b (cx+y) where c=a/b.

andy.fingerhut10:01:21

Also fewer arithmetic operations to do with just ax+y, so faster.

qqq10:01:16

if ax+y is faster than ax+by, than I would have expeced blas to offer x+y, ax+y, ax+by as three different functions

sveri13:01:53

@qqq where some of the few bugs / features were added by FooCorp? How would an applicant know this was not the case?

sveri13:01:32

Also depending on the code size it might take a lot of time to work into a project. Now imagine being in a application phase where I apply to like 10 corporations, have my current job and a family and every corp expects me to fix a few bugs in some random project? I would have to be very very very very desperate to do that.

sveri13:01:00

Regarding "DSL -> wasm". I wonder how you would do that, wasm is stored in a binary format, so do you want to write a compiler that emits that binary format?

qqq13:01:08

@sveri: no, the 'list of open source projects' would be things like Cider, Reagent, ... other popular clojure libraries

qqq13:01:13

The situation is this: as a Founder, I don't want to interview 100 people to figure out who's competent, I'd rather tell them: Hey, those of you that claim to be proficient at Clojure -- we use the following 10-20 popular clojure libraries -- go to their github page, start reading some public github issues, and fix a fewsof them.

qqq13:01:05

(For those who already have open source contributions, they can just list their existing commits.)

qqq13:01:14

The problem this aims to solve is: people lie / overstate on their resumees all the time. So, a simple "proof of competence" would be: if you're good at clojure AND you can work with other people AND you are a self-motivated hard worker then go to this lsit of popular clojure libs, read a few github issues, and fix them -- until you get 100 lines committed (and you would only have to do this once -- and there aren't Company specific bugs -- they're clojure lib specific bugs)

bronsa13:01:34

yeah that’s hardly going to work

bronsa13:01:52

people in general don’t like being required to do free work in order to be eligible for an interview

qqq13:01:45

they're not doing work for the company; they're doing work for the open source community

bronsa13:01:27

that’s still free work for the individual

bherrmann13:01:51

Back in the 90s, we would bring in C++ candidates. We would hand them a sheet of paper with two problems. Write some code to sort an array. Write some code to do factorial n. Take as much time as you want, use this confrence room. People would just walk out immediately, or after about 10 minutes leave....

qqq13:01:11

@bherrmann: how did they get by the phone interview ? 🙂

qqq13:01:03

@bronsa: let's take you for an example, have you contributed more than 100 lines to clojure ?

bronsa13:01:10

but on my own free will

bherrmann13:01:13

People talk a lot of smack - but you are right ... I would see people come in and go... I wasnt part of the process, my managers were. Presumably they spoke well.

bronsa13:01:14

nobody asked me to do it

qqq13:01:26

we would accept past contribs too; see, this is a wonderful test!

bronsa13:01:31

and most of my open source contributions have actually been paid

bronsa13:01:38

right but what if I hadn’t

bronsa13:01:07

imagine I had the same skillset as I have now, but hadn’t contributed any lines to clojure

bronsa13:01:47

I’m certainly not willing to spend hours/days of my time to contribute to oss just to be eligible for an interview, and so aren’t a big chunk of developer in the pool

sveri13:01:15

I dont think so. The open source community lives in a bubble and if you do that process, you will exclude like 95% of other developers. I am not saying this is a bad thing, its a decision you make. In the last two years we had approx. 15 applicants and exactly 0 of them did open source contributations in their past. And if we would have asked them to do some they would have politely declined.

qqq13:01:59

well, this isn't our team going out and emailing senior engineers and saying: hey, make a contrib to open source to interview with us

bherrmann13:01:15

https://github.com/clojure/clojure has 141 contributors... thats a pretty small pool to draw from...

qqq13:01:18

this is us stating for our hiring policy (to people who send their CVs to us): hey, prove your worth by making open source contribs

bronsa13:01:49

@qqq right, and I’m saying that’s not in your interest if your interests are actually getting candidates

bronsa13:01:08

you’ll just be drawing candidates away

qqq13:01:21

@bherrmann: there are other libraries, boot, lein, cider, criterium, transit-clj, ring-core, portkey, etc ...

bronsa13:01:51

I think you’re overestimating the % of developers with non-minor OSS contributions

bherrmann13:01:59

For my prior gig, I had to solve a problem after they phone screened me. It was work, but it was also fun. And we had something to talk about during the interview... They used the same puzzle with most candidates, so they knew the ins and outs well.

bronsa13:01:27

there’s a difference between solving interview problems and being asked to do OSS work for free

bronsa13:01:45

the OSS work you’re doing is highly likely to indirectly benefit the company

sveri13:01:49

Also 100 lines of quality code will be like 20h of work. Given my current situation / spare time I could do that in approx. 1 month. And thats for one company. This wont be an option for me.

bherrmann13:01:09

I did spend 6 hours on the solution, which they expected would take most people 2 hours. But I wanted the job.

bronsa13:01:11

right, but I don’t even see it as a matter of time, even if the OSS contribution they asked you to do took less than 6 hours.. one is a coding exercise, the other is unpaid work

qqq13:01:54

large teams have increased overhead due to commuhnication complexity; say that instead of a team of 10 engrs * 150k each you decide 3 "10x" engrs * 500k each I suspect there's a lot of people who would jump through the OSS hoop for a 500k salary but not for a 150k salary

dpsutton13:01:08

sure. but you're not going to find 10x engineers with coding examples and applications. those are happily employed and you need to seek them out

sveri13:01:27

If they exist at all.

bronsa13:01:33

also 10x engineer fallacy

dpsutton13:01:39

also, a team of "10x" engineers might not be as great as the thought on paper

qqq13:01:50

well, time will tell, could be a silly hill to have a startup die on 🙂

bherrmann13:01:57

I'd probably try to create a good team... I'm a leaf node still, but I'd probably want to follow up on this research, https://www.inc.com/michael-schneider/google-thought-they-knew-how-to-create-the-perfect.html

vemv13:01:18

don't think 10x is a fallacy (don't want to open that can of worms though!) in case anyone appreciates the observation: IME, the 10x-ness comes from a greater skill in detecting/satisfying the need for abstractions. but probably you don't want a team where everyone constantly creates abstractions and 10xers typically aren't happy just churning out features? so one should aim for a balance in profiles.

cassiel13:01:58

For anyone interested who’s in London this weekend: we have a big public art AV sculptural piece opening on Thursday evening, running until Sunday. Visuals are all driven live via ClojureScript (plus THREE.js and friends). Do come and say hello… http://cassiel.com/2018/01/06/cosmoscope-in-london/

sveri13:01:04

Has anybody ever seen a 10x?

cassiel13:01:21

(Sorry - I seem to have dumped my post into the middle of a running conversation.)

vemv14:01:32

Worth remembering that '10x' comes from a Lisp essay 🙂 or can't a Clojure team outperform 10x an OO one? (for tough problems at scale)

chrisblom14:01:28

10x in what? speed? maintainability? clarity? lines of code?

vemv14:01:09

I'd summarize it as effort-outcome ratio

sveri14:01:39

and what is the outcome?

vemv14:01:40

a system with a set of subjectively-measured properties (does it meet goals, is it maintainable)

jgh15:01:56

i think the thing that separates “10x” from “1x” I guess is just being a self-starter and follow-through. A lot of devs you have to give them instructions for every little thing. Which is fine when you’re talking about a company with a decent amount of resources and time available. Spec out every detail. Great. But if you’re talking about small companies where people need to be more autonomous and make their own decisions you dont want to have to sit there and give someone all the minutiae lest they fuck it up.

jgh15:01:17

Will you have tech debt after something like that? Probably, but if you’re trying to move quickly that’s a tradeoff you’ve already decided on.

roberto15:01:45

so it is just someone who write sloppy code at breakneck speed

jgh15:01:03

No, that’s a false equivalency

jgh15:01:12

fast != sloppy.

jgh15:01:31

Conversely, slow != thoughtful

fellshard15:01:29

Some of it's correlated with drive to learn, some with how rigid/immature your model of computing is, what disciplines you learn and apply, etc.

scriptor15:01:50

everyone has their own definition of what a 10x dev means

fellshard15:01:04

In general, it's not a categorical statement - you can learn to be a 10x developer; conversely, you can become lax and drop out of the 10x status

jgh15:01:16

Sure, in some cases one person may be 10x…in other cases the may be 0.5x 😉

fellshard15:01:38

Put me on UI and I'll show you just how 0.1x I can be 😛

jgh15:01:59

oh yeah im the same way, UI is the worst and a definite fast track to 0.1x

jgh16:01:14

people who love it though love creating experiences for the user….i get that motivation but i just can’t care that much about magical experiences or something lol.

scriptor16:01:19

@jgh what kinds of things did you mean devs might need to be instructed on?

scriptor16:01:48

it's one thing if a developer needs fleshed out product specs/requirements, but anyone who's not a junior shouldn't need handholding in terms of the actual engineering

roberto16:01:25

10x == young impressionable , who in their desire to please/show off will disregard their health

jgh16:01:55

i just mean some people need to be given more detail on intermediary steps of the operation of the system. I don’t think it’s a bad thing. If you can follow specs/requirements that’s a good trait to have. But some environments don’t really have specs to the extent that some people seem to need it.

jgh16:01:25

(i’m speaking mostly from my experience in working in startups)

scriptor16:01:53

what do you consider an intermediary step?

scriptor16:01:25

a product person shouldn't be dictating the technical specs, whether it's at a startup or a large corp

scriptor16:01:40

requirements are another matter, though

jgh16:01:31

im trying to think of a good example.

jgh16:01:58

like if for example I want to take inputs A, B and emit output C and we have a codebase with a bunch of transformations that can do different things and are basically plug and play

jgh16:01:11

i dont want to have to sit and tell you which components to use in which order and why

jgh16:01:14

i want you to figure it out

jgh16:01:01

read documentation, read code, read standards (if you have to build a new thing)

jgh16:01:27

now if i had more money and time we could sit down and do architecture meetings and have more documentation and specification than code.

jgh16:01:26

but since this is the startup world and we’re trying to “move fast and break things” that’s a luxury we dont necessarily have 😛

scriptor16:01:23

fair enough, I'd expect someone with a good bit of experience (at least a few years) and knowledge of the system to pick that up

scriptor16:01:19

although we've found that even a quick "design review" of their proposed architecture helps a lot

jgh16:01:55

yeah that does help.

sveri16:01:18

So I think a better definition would be 10x = a "person" that has in depth experience in the task that is handed to him and that is able to fullfill that task alone in a reasonable speed with a reasonable low enough error count.

scriptor16:01:34

to me that's what I'd expect of any engineer who has worked in a given domain for enough time

fellshard16:01:45

And yet, a good chunk don't, so spending time in the domain alone isn't sufficient; what's the primary factor?

jgh16:01:46

basically the 10x thing is a myth heh

fellshard16:01:07

It's not so much about literal '10x' and more an observation of an order of magnitude in difference between how some devs are able to model and navigate a system compared to other devs, and I suspect it's not universal - many of us here could probably delve into a functional system with ease, whereas other people might be able to wrangle a more stateful OO system more easily, etc.

scriptor16:01:49

I wonder if people had similar conversations 10 years ago about what exactly a "rockstar ninja" dev was

fellshard16:01:13

Rockstar ninja is a myth of personality

jgh16:01:15

well being a software dev is good pay compared to a lot of other jobs so I guess if you can get by doing the bare minimum for a career and dont have higher aspirations than that (e.g. your primary motivations in life are not really programming-related or work-related at all) then it’s not a bad place compared to getting a job at an insurance company or something.

jgh16:01:21

but if that’s how you go through life certain environments won’t be good for you, like startups or highly competitive tech firms.

fellshard16:01:31

It is, but it's not tenable for the industry in the long run as it erodes the reputation of developers and the expectations for the type of work you can give them.

jgh16:01:44

maybe, but there are lots of places that can get by on mediocre devs plugging away at C# day in and day out.

fellshard16:01:02

And watch the mess their systems will be in over time. The debt accumulates.

fellshard16:01:20

This is why I tend to get on board with the software craftsmanship folks.

tcarls16:01:49

I've absolutely worked with folks who have that kind of productivity multiplier -- the successful startups I've been at (including current post-acquisition one) have at least one at the core. Which typically involves understanding enough of a problem space to come up with a new and different way to slice/abstract/analyze an issue -- just one of those insights is what created my current company's entire technical advantage over its competitors.

jgh16:01:53

i worked for a government contractor for a spell there and in my first week I found a “bug” where the person who wrote the code used “sleep(1)” instead of “thread.join()” and I sped the system up by like 1000%. Why they were spawning that many threads in the first place was another issue, but you can see where im going with this.

tcarls16:01:11

btw, personally, I've made a lot of OSS contributions while working for various employers, to the point that it generally surprises me when someone hasn't made any.

fellshard16:01:29

It reaaally depends on how the employer looks on that kind of thing

tcarls16:01:34

But then, I have a history of working in positions where there's a lot of system integration going on -- gluing together various 3rd-party components &c

fellshard16:01:39

As a consultant, as a general rule it's a no-go

dpsutton16:01:46

i know someone in silicon valley who is prevented from doing any open source work

tcarls16:01:50

so making those 3rd-party components work right is critical.

tcarls16:01:02

OSS == open source software

jgh16:01:12

no i know that, i meant what fellshard was asking

jgh16:01:16

err referring to

fellshard16:01:20

One client hired us to make an open source client library for their API, so there's cases where it happens, but it's tricky

fellshard16:01:49

Context switch!

tcarls16:01:57

My first company that did a significant amount of consulting was MontaVista Software -- so there, we were maintaining Linux dev stacks for various embedded systems.

jgh16:01:10

i have a couple of open source things that i specifically carved out of my latest contract, and most of the other stuff i’m doing is clojure/docker-related right now so nothing to do with the contract heh

tcarls16:01:19

a client could pay us to maintain custom local patches that were unreleased, but that was a lot more expensive than building something once and getting it integrated upstream for the community to maintain.

tcarls16:01:17

and a lot of ie. the build-system work I did involved building patches to various packages to make them cross-compile. Whenever those patches were to GPLed software, we had to distribute the source under GPL terms to anyone who got a copy of the binaries, so... well... there you are.

jgh16:01:04

has anyone ever been sued over an OSS license?

tcarls16:01:15

jgh, yes, plenty of times.

jgh16:01:18

I know, e.g. ffmpeg keeps a shit list

tcarls16:01:30

the nastiest such cases being over really poorly written licenses. There was a model train control system in particular that was somewhat infamous -- the person who broke the license's intent by building a commercial product without crediting the author / distributing source / etc. actually ended up countersuing the original maintainer (and going after their employment -- it was nasty)

tcarls16:01:16

but there are also a lot of GPL enforcement cases in Germany -- law firms there are allowed to file suits in the public interest, so they don't need a copyright holder who's been aggrieved to step up and actively act as plaintiff, as is typically the case elsewhere.

jgh16:01:30

interesting

tcarls16:01:57

...if a corporation infringes the FSF's copyright, they typically aim for a settlement that includes agreement to comply with the license in the future and a monetary donation, and those settlements generally happen fairly quickly.

joelsanchez16:01:15

( apparently it's this one if you want to know more https://en.wikipedia.org/wiki/Jacobsen_v._Katzer )

vemv17:01:38

never mess with a train modeler

noisesmith21:01:06

the app is made in clojure and clojurescript, of course

noisesmith21:01:05

another fun one is that we did a network on scala at one point, and rich was rated as #50 out of 100 people in terms of influence (measured by the significance of his placement in a graph of connections)

fellshard21:01:34

Surprisingly pervasive

noisesmith21:01:08

I’m running a new scala network with 1000+ nodes this time to compare (the old data was ~2 years old)

noisesmith21:01:54

he is not in this one, so the old one could have been a bug or a statistical blip

noisesmith21:01:17

he’s followed by 25% of the network, but he isn’t in the network as he doesn’t follow anyone in the network - so either our algorithm for inclusion is better now or he got pickier

timrichardt23:01:44

@noisesmith could you elaborate a bit on this graph?

timrichardt23:01:04

what is shown, what do the colors mean

noisesmith23:01:34

it’s a social connection graph, there’s a partition algorithm (finding groupings that maintain the most internal connections) that finds subgroups

noisesmith23:01:53

so each color is a subgroup (aka clique or community) within the larger group

noisesmith23:01:14

simple version is they have more connections to one another than they have to the rest of the graph

noisesmith23:01:55

then we use graph analysis to figure out relative influence based on position in the network - you are more influential if you are in more shorter paths - if more people are connected via their connection to you

timrichardt23:01:39

how you determine the nodes, am i a node? and how you define a connection?

noisesmith23:01:08

we use social data - in this case users on twitter because twitter is open about their data and has convenient accessible apis

noisesmith23:01:17

so the connections are follows

timrichardt23:01:32

ah, ok. nice 🙂

timrichardt23:01:57

do you think its representative?

noisesmith23:01:19

representative of the app and algorithms, representative of the clojure community?

noisesmith23:01:07

in the ranking, there’s all the suspects you’d look for, hickey, nolen, sierra

noisesmith23:01:27

I was surprised to see miller so low, but I think he’s less of a twitter user

timrichardt23:01:57

what if rich had no twitter account

timrichardt23:01:16

ok, what if questions are stupid...

timrichardt23:01:34

i am just impressed 🙂

noisesmith23:01:35

the data would be less informative, because a lot of people are found by looking at who follows rich, who rich follows, etc.

noisesmith23:01:02

and then keeping people that have more dense connections (both going out and coming in), throwing away the rest

noisesmith23:01:47

thanks, I’m proud of our work, it’s part of a larger SaaS business suite now - not the best outcome for a startup, but far from the worst

noisesmith23:01:10

the first version was ruby, and the clojure rewrite was a big risk but I think in the big picture using clojure was a net benefit

timrichardt23:01:08

yes, also lets thank the twitter api

timrichardt23:01:24

is it publicly available? the app.

noisesmith23:01:32

indeed - it’s the substrate without which we’d have nothing to offer

noisesmith23:01:41

it’s a SaaS - so no, sadly

timrichardt23:01:44

do you aim other platforms?

timrichardt23:01:48

ah, overlooked.

noisesmith23:01:54

it’s part of an enterprise suite called Sprinklr

noisesmith23:01:41

we have plans to support other platforms, but there’s been more potential in expanding features instead of using more limited data sources

timrichardt23:01:08

anyway, i would like to see an investigation on the different groups in your graph

timrichardt23:01:19

i think this would make a nice little talk

noisesmith23:01:20

it’s hard to find good firehoses of data, but for enterprise customers we might end up processing their existing internal data

noisesmith23:01:42

@timrichardt cool - yeah, the graph is interactive, made with d3

timrichardt23:01:58

who are they, 9 different groups

noisesmith23:01:12

one moment I’ll screenshot that view

noisesmith23:01:50

a big part of the utility for our end users is that they can say “what are clojure people talking about” and see related topics that are actually coming up organically, which is a lot of insight if you are eg. planning a conference or trying to do market research and developing a product or even just trying to recruit

noisesmith23:01:10

(substituting any other topic for clojure of course)

timrichardt23:01:47

that is impressive.

noisesmith23:01:54

when testing the app, the best test is a topic I already know well, so I can sniff test “are those really the most relevant people to know about and follow?”

timrichardt23:01:58

i really would like to have this detailed out and like to have a comment from people who observe the clojure community alot.

timrichardt23:01:26

exactly, my feeling is, twitter is not enough 🙂

timrichardt23:01:43

but if you add github, the meetup pages etc...

noisesmith23:01:33

well, those would help this specific search, but it wouldn’t really help $CORP who wants to know what’s going on in the world of $WIDGETFANS

noisesmith23:01:06

for them it might be pinterest, or instagram, or some php bulletin board site

noisesmith23:01:31

also many sites (including twitter) have a TOS with their data that you aren’t allowed to cross reference identities with other datasets

timrichardt23:01:01

i am uploading my pulse hourly to samsung, i dont worry about all the traces we leave. i hope this moderately expands and i am always happy to see outcomes of all this recordings like yours. i think on the long term we will discover alot of new stuff with this.

timrichardt23:01:43

but it is also scary, so easy to abuse at the moment.

noisesmith23:01:21

yes and there are definitely people abusing these techniques already (eg. using services to doxx people by cross referencing activity between datasets)

noisesmith23:01:21

worst outcome with our app is that you get targeted ads that are about topics you are apparently interested in already, or an annoying recruiter who knows about your skills or interests

noisesmith23:01:46

well - maybe there are worse things possible but given the business model they aren’t probable?

timrichardt23:01:43

i guess the business model is exactly those things have to be probable 🙂

timrichardt23:01:15

no recruiter would consult you, if he would learn nothing about me

timrichardt23:01:45

but thats a large topic

timrichardt23:01:52

and its late for me 🙂, my bed is calling