How to Handle End of Stream

[ssadedin]

I have a few common use cases that I'm finding hard to deal with. The most obvious one is I want to process lines of a file. It's all fine except that I don't know how to ensure the file is closed once the stream is finished. If I pass Stream objects around I have no way to ensure that. Intuitively, what I want is a syntax like this:

def foo() {
    def r = new File("test.txt").newReader()
    return new Stream({ f.readLine() })
        .filter { blah }
        .map { bloop }
        .end { r.close() }
}

Then a client of this function can call:

foo().map { fizzle }.until { blag }

And the key point is, the "end { }" closure is guaranteed to be called when the stream terminates, regardless of whether it's because the file was exhausted or whether it's because someone downstream imposed an "until".

Is there a way to handle this right now, or is there a possibility to add such an "end" function?

[timyates]

Hiya!

Don't think there's a way to do this currently :-(

If you can do all the processing inside foo then obviously you could just
use withReader, but that doesn't help with this use case

I'll have a think, I can't think of a similar method in other
iterator/stream frameworks and wonder if there's a reason.

Tim
On 25 Jun 2015 03:01, "Simon Sadedin" [email protected] wrote:

I have a few common use cases that I'm finding hard to deal with. The most
obvious one is I want to process lines of a file. It's all fine except that
I don't know how to ensure the file is closed once the stream is finished.
If I pass Stream objects around I have no way to ensure that. Intuitively,
what I want is a syntax like this:

def foo() {
def r = new File("test.txt").newReader()
return new Stream({ f.readLine() })
.filter { blah }
.map { bloop }
.end { r.close() }
}

Then a client of this function can call:

foo().map { fizzle }.until { blag }

And the key point is, the "end { }" closure is guaranteed to be called
when the stream terminates, regardless of whether it's because the file was
exhausted or whether it's because someone downstream imposed an "until".

Is there a way to handle this right now, or is there a possibility to add
such an "end" function?

—
Reply to this email directly or view it on GitHub
#21.

[ssadedin]

Thanks for the followup - yes, I'm not sure what the right solution is. TotallyLazy seems to have some functionality along these lines, but I can't see any examples of how it works - see:

https://github.com/bodar/totallylazy/blob/master/src/com/googlecode/totallylazy/Closeables.java

JDK8 streams seem to sidestep the issue by putting a "close()" method onto the Stream interface itself:

https://docs.oracle.com/javase/8/docs/api/java/util/stream/BaseStream.html

But that just means the consumer of the stream has to close it. If nobody else is doing it then It might be that what I want is an anti-pattern. But it seems intuitive to me that being able to functionally compose streams is a desirable thing.

[timyates]

I didn't realise this was a thing with Java 8 streams :-)

I'll take a look into adding onClose and close into Groovy streams... As
they are just a chain of iterators, it should be possible to walk up the
parent, and silently try to close any resources that have been captured...

Will need to make sure we don't kill auto-closeable AND grooy's
withReader/InputStream though ;-)

Tim

On 25 June 2015 at 13:38, Simon Sadedin [email protected] wrote:

Thanks for the followup - yes, I'm not sure what the right solution is.
TotallyLazy seems to have some functionality along these lines, but I can't
see any examples of how it works - see:

https://github.com/bodar/totallylazy/blob/master/src/com/googlecode/totallylazy/Closeables.java

JDK8 streams seem to sidestep the issue by putting a "close()" method onto
the Stream interface itself:

https://docs.oracle.com/javase/8/docs/api/java/util/stream/BaseStream.html

But that just means the consumer of the stream has to close it. If nobody
else is doing it then It might be that what I want is an anti-pattern. But
it seems intuitive to me that being able to functionally compose streams is
a desirable thing.

—
Reply to this email directly or view it on GitHub
#21 (comment)
.

[timyates]

So, I had some free time tonight, and took a stab at adding a close() method to streams to try and close all sources being used by the stream...

It ended up being a bit of a beast

Got all existing tests passing, and added a solitary test for the new close method

But as the commit message says, this commit feels like it needs a lot of looking at until it can be trusted...

I guess you were more looking for an onFinished method though, to attach a closure/Function to be run when the stream has concluded?

which I guess is another commit, on another free night ;-)

[ssadedin]

This looks like a great start, and certainly helps to manage the issue. I
have cloned / built the feature branch and will try and exercise it over
the next few days. It worked for my simple tests already.

One minor note - this made it depend on Java 1.7 for me. Not a big deal, I
think, but until now I could run it on JDK1.6, so thought I'd mention it in
case that's important.

A generalised "onFinished" type feature would definitely still be a great
thing to have!

Thanks!

On Mon, Jun 29, 2015 at 6:49 AM, Tim Yates [email protected] wrote:

So, I had some free time tonight, and took a stab at adding a close()
method to streams to try and close all sources being used by the stream...

It ended up being a bit of a beast
7242e70

Got all existing tests passing, and added a solitary test for the new
close method

groovy-stream/src/test/groovy/groovy/stream/ReaderTests.groovy

Line 75 in 7242e70

def "Check the reader is closed if the Stream is closed"() {

But as the commit message says, this commit feels like it needs a lot of
looking at until it can be trusted...

I guess you were more looking for an onFinished method though, to attach
a closure/Function to be run when the stream has concluded?

which I guess is another commit, on another free night ;-)

—
Reply to this email directly or view it on GitHub
#21 (comment)
.

[timyates]

Cool, thanks for trying it all out :-)

Yeah, adding AutoCloseable (so it can be used in a trywithresources query),
has pushed it to Java 7... My main thought is that as Java 7 is already
EOL, that's fine, but I guess others may disagree, and it may reduce the
possible userbase

I'll have a look at onFinished :-D

On 30 June 2015 at 14:07, Simon Sadedin [email protected] wrote:

This looks like a great start, and certainly helps to manage the issue. I
have cloned / built the feature branch and will try and exercise it over
the next few days. It worked for my simple tests already.

One minor note - this made it depend on Java 1.7 for me. Not a big deal, I
think, but until now I could run it on JDK1.6, so thought I'd mention it in
case that's important.

A generalised "onFinished" type feature would definitely still be a great
thing to have!

Thanks!

On Mon, Jun 29, 2015 at 6:49 AM, Tim Yates [email protected]
wrote:

So, I had some free time tonight, and took a stab at adding a close()
method to streams to try and close all sources being used by the
stream...

It ended up being a bit of a beast
<
7242e70

Got all existing tests passing, and added a solitary test for the new
close method
<

groovy-stream/src/test/groovy/groovy/stream/ReaderTests.groovy

Line 75 in 7242e70

def "Check the reader is closed if the Stream is closed"() {

But as the commit message says, this commit feels like it needs a lot of
looking at until it can be trusted...

I guess you were more looking for an onFinished method though, to attach
a closure/Function to be run when the stream has concluded?

which I guess is another commit, on another free night ;-)

—
Reply to this email directly or view it on GitHub
<
#21 (comment)

.

—
Reply to this email directly or view it on GitHub
#21 (comment)
.