NYCPHP Meetup

[Anthony Wlodarski]

Brent,

Thanks for the feedback, we are going to do the XML file in batches but it is interesting to see that the memory from within the foreach never gets returned (tested with memory_get_usage).  Now there are a couple of other variables to assume with this application.  It is Drupal based so yeah there is bloat but we take that into consideration (our memory size is set to 128mb per operation) and we also assume that the postgresql db driver in Drupal is doing its job on not creating un necessary connections for each call to node_load() but we haven't dug that deep into the Drupal core for times sake.  But I didn't know you could use the variable in a foreach as a reference, I will have to take that into consideration when being a memory conservationist in the future.

-Anthony
--
Anthony Wlodarski
www.thrillist.com <http://www.thrillist.com>
Web Applications Developer
568 Broadway Ste. 605
New York, NY, 10012
(o) 646.786.1944


________________________________
From: Brent Baisley <brenttech at gmail.com>
Reply-To: NYPHP Talk <talk at lists.nyphp.org>
Date: Mon, 26 Jan 2009 09:50:38 -0800
To: NYPHP Talk <talk at lists.nyphp.org>
Subject: Re: [nycphp-talk] PHP Scope in foreach construct.

foreach makes a copy of the array element you are processing. If the
current "value" is large, like another array, then you have 2 large
arrays. Use a for loop instead or use a reference for the value
variable:
foreach ( $array as &$value ) ...

If you are running into memory problems, you probably should rethink
how you are processing the data. You should process it in chunks and
stream it out to the XML file. Then it does matter how large the data
set gets.

You can also just adjust the php memory limit if you haven't done that
already. It defaults to 8MB or 16MB (5.2).

Use memory_get_usage to see where your memory is jumping.

Hope that helps.

Brent Baisley

On Mon, Jan 26, 2009 at 12:27 PM, Anthony Wlodarski
<anthony at thrillist.com> wrote:
> When working with foreach constructs do the functions and variables defined
> get returned to the garbage collector on each iteration or does the foreach
> have to finish execution before things can be returned to the garbage
> collector?
>
> The reason that I ask is because we are generating data for Google Maps and
> they require one very large XML file which keeps blowing our memory size
> definition inside the php.ini file.  What we might end up doing is creating
> the XML file in batches of lets say 500 items or less and then append the
> files manually.
>
> I figured this was an interesting question and I can't find any answers
> anywhere else (even on php.net docs).
>
> -Anthony
> --
> Anthony Wlodarski
> www.thrillist.com <http://www.thrillist.com>
> Web Applications Developer
> 568 Broadway Ste. 605
> New York, NY, 10012
> (o) 646.786.1944
>
> _______________________________________________
> New York PHP User Group Community Talk Mailing List
> http://lists.nyphp.org/mailman/listinfo/talk
>
> http://www.nyphp.org/show_participation.php
>
_______________________________________________
New York PHP User Group Community Talk Mailing List
http://lists.nyphp.org/mailman/listinfo/talk

http://www.nyphp.org/show_participation.php

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.nyphp.org/pipermail/talk/attachments/20090126/77eff66d/attachment.html>