Seems more like a systematic challenge rather than a syntax one. Not sure how adding some kind of character will stop behaviour without discussing the underlying issue Maybe just talk to the team and see what would help them

I wouldn't resolve to some "easy" convention in such a case. ... is ambiguous (even if it was working), -> and ! are overloaded. If you really want to try solving it with naming, I'd use results-LL ("lazy and large") or results-On (for O(n)). Or even go verbose and use large-and-lazy-results. A more robust solution IMO would be to wrap that collection and in all relevant methods check if some dynamic variable is set, then set that var in a macro so it ends up being something like

(with-awareness-of-large-results
  (count results))

Thanks - interesting approaches. @U05254DQM - agree that this is a team challenge and we should just talk about it. @U2FRKM4TW - verbose naming probably the pragmatic way to go but your robust solutions are interesting too. Perhaps use of sequences is simply so idiomatic to Clojure that folks really need to learn that style, rather than hand-holding them too much by adding macros etc... That said, closing over the cursor and only allowing you to pass a function to map over might work for me ....

Hmm ... except you could pass it identity and get back the cursor again.... Perhaps close over with a function that applies dorun to any (side-effectful) function passed in? Seems yuck but ...

> except you could pass it identity and get back the cursor again That solution is, in my mind, not intended to prevent anything. It's intended to guard in a much more explicit way than just naming and to add a potential barrier in front of doing something bad.

Not sure if it fits your exact pattern but using reducible queries helps makes this distinction. The transducer context can be quite nice to kinda keep things local and iterate once

I use the ! pattern to denote "be careful", meaning go look at the doc for it. If it was a local it be a comment instead of a doc-string. And then the comment/doc-string would say that results can contain a very large N, something like that.

But not everyone uses ! to mean that.

You could also go real verbose if you're really worried a out it: resutls-do-not-hold-on-to-head or results-is-very-large-coll-be-careful

Also, just a comment might be enough

You can require a LIMIT clause on the sql cursor statement. If someone writes code that specifies LIMIT 3 then it's ok to count the results. But if they say LIMIT 1000000000000000000000000000000000000000000000000000000000000000000 then they should know to tread carefully.

What are you calling at the REPL?

And perhaps something that could illuminate, what do you think should happen if you called (log-email []) from the repl?

❯ clj
Clojure 1.11.1
user=> (.getTos [])
Execution error (IllegalArgumentException) at user/eval1 (REPL:1).
No matching field found: getTos for class clojure.lang.PersistentVector
user=>

@U11BV7MTK edited the question. I am running the project with lein repl and a frontend action is calling the function

that’s what i figured. Does my example give any help?

You’re calling log-email with an instance of the class com.sendgrid.helpers.mail.Mail. What you’re importing in the ns doesn’t really matter here.

You say that "`getTos` is defined in ...`Personalization`" . Perhaps that is where the current problem is, and has nothing to do with this function at all? Somewhere, in your project, a piece of code that is getting evaluated tries to call .getTos method of a com.sengrid.helpers.mail.Mail, which doesn't have such method. That would not be happening in the log-email function unless you are actually calling log-email. The code shown so far never actually calls that function, so you are not showing us what inappropriate value is being passed into the function.

@U11BV7MTK yes

https://gist.github.com/mk/d8f60f0b13bf6a63c6afc89d6f7e7565

I guess it's related to this hack:

;; HACK
(defn -deftype [name class-name args interfaces]

  (doseq [arg [class-name name]]
    (memo-clear! members* [arg])
    (memo-clear! members* [(str arg)]))

  (let [interfaces (mapv #(symbol (.getName ^Class %)) interfaces)]
    (eval (list `let []
                (list 'deftype* name class-name args :implements interfaces)
                (list `import class-name)))))

iirc it's a hack relating to how analysis works

I guess the hack could be avoided when the type already exists and matches the code that's being analyzed?

@U5H74UNSF This is a hack which avoids the hack:

(do
  (let [form '(defrecord MyRecord [form])
        old-deftype-hack ana-jvm/-deftype]
    (with-redefs [ana-jvm/-deftype (fn [name class-name args interfaces]
                                     (when-not (resolve class-name)
                                       (old-deftype-hack name class-name args interfaces)))]
      (ana-jvm/analyze form (ana-jvm/empty-env) {}))) :analyzed)

@U060FKQPN There is no clean way to customize the deftype* analysis to avoid that eval side effect right?

not really, FWIW it's something that the clojure compiler also does internally, it's related to having the ability to correctly do type analysis on deftype methods

You could check if the class already exists, no?

well, ish

it must exists and match the definition under analysis

if anybody wants to make a ticket I could spare some time to have a look at it, unless somebody also wants to provide a patch :)

Yeah Which you can't really verify in a repl

you can

How? You could check the signatures match, but the implementation?

does the impl matter for analysis?

yeah sorry, I said implementation but I mean interface

wait, I said neither 😄

but yeah, the impl doesn't matter. just the signatures of the methods it implements

So it should be possible for deftype, but not for reify, right?

we could also have an option to "assume that a same-named class will have been evaluated with the same form before" or so

no real difference between deftype and reify at that level

as to not overwrite existing impls which isn't great for working in the same environment

another approach would be to prefix the class used for type analysis, but then we need to be careful to rename things correctly everywhere

... yet another approach would be to define this class in an isolated classloader

yes

FYI I have a comment here that may be a bit annoying to hear, but: this in general should not be a problem that exists, because t.a. fundamentally works with analyze+eval, not just analyze

but ok, there are options we can consider to make this a bit nicer

just not one that's currently implemented I'm afraid

I think you could do the analysis binding clojure's classloader to an isolated one

and that may make the redefined class not leak into your classloader

I can have a go at an example in a few hours, a bit busy now

:bindings {Compiler/LOADER ...} yeah, just gotta figure out how to do this

I guess eval doesn't play well with the Compiler/LOADER binding? https://github.com/clojure/clojure/blob/527b330045ef35b47a968d80ed3dc4999cfa2623/src/jvm/clojure/lang/Compiler.java#L7139

makeClassLoader creates a new DynamicClassLoader based off of Compiler/LOADER

aha

the classCache will still be a problem though

so yeah, not straightforward

there are hacks around it

I wonder why this doesn't crash:

(with-bindings {clojure.lang.Compiler/LOADER (proxy [ClassLoader] []
                                                 )}
    (eval '(deftype Dude1 [])))

user=> (defmacro hack-loader [] (var-set clojure.lang.Compiler/LOADER nil))
#'user/hack-loader
user=> (eval '(let* [] (hack-loader)))
Syntax error (NullPointerException) compiling fn* at (REPL:1:8).
Cannot invoke "clojure.lang.DynamicClassLoader.defineClass(String, byte[], Object)" because "this.loader" is null

here's a very hacky way to set the loader

potemkin has some custom deftype/defrecord like macros that attempt to avoid re-defining classes if they already exist, and people also run into problems with that behavior

https://clojurians.slack.com/archives/C03S1KBA2/p1656690868855769

@U060FKQPN couldn’t quite figure out yet how to pass an isolated classloader to tools.analyzer yet. Do you have more pointers for me? 🙏

also wondering if you could expand a bit on your comment regarding tools.analyzer fundamentally working with analyze+eval. I understand that any macros need to be evaluated before so analysis can use them, same for fns used by macros. Beginning to think the same is true for protocols / deftypes.

I don't have a good working suggestion for you yet, give me a few days, I'll try to get you something working on the weekend

thanks a ton for your help!

@U5H74UNSF currently the only suggestion I have for something that can work for you would be to with-redefs eval to be a no-op during analyze. (i.e. (with-redefs [eval (fn [x] nil)] (ana.jvm/analyze ..))) I am exploring options but it will require some time

as per my comment wrt analyze+eval -- yes, your idea about marcors needing to be evaluated + all their dependency tree is correct, and that's the main point. and for deftypes we also need to do this to proved correct method resolution. it is also however true that if 100% analysis accuracy is not required, t.a does expose some hooks where the user can plug in and provide escape hatches to avoid needing full evaluation, and this one seems like it could be a good candidate for one

makes sense, thanks for clarifying