From using MySQL, I’ve used the ‘unbuffered queries‘ feature a number of times. It’s where you don’t fetch the entire resultset into memory at once – which is necessary if you’re retrieving more data than you have memory available. If’s often also generally gets results/data back to you sooner.
Pseudo PHP code could be a bit like :
$pdo = new PDO("mysql:host=whatever;dbname=whatever", $user, $pass.);
$pdo->setAttribute(PDO::MYSQL_ATTR_USE_BUFFERED_QUERY, false);
$stmt = $pdo->prepare("SELECT * FROM big_table");
$stmt->execute();
while($row = $stmt->fetch(PDO::FETCH_ASSOC) ) {
echo $row['field'];
}
Today, I found myself needing to do the same but with PostgreSQL, and it’s been some time since I used it in anger… so had to do some research.
Unfortunately, PostgreSQL doesn’t have the exact same behaviour available (so I can’t just change the PDO DSN/connection info), and I ended up discovering CURSOR queries – example PHP code below.
$curSql = "DECLARE cursor1 CURSOR FOR SELECT * FROM big_table";
$con = new PDO("pgsql:host=whatever dbname=whatever", "user", "pass");
$con->beginTransaction(); // cursors require a transaction.
$stmt = $con->prepare($curSql);
$stmt->execute();
$innerStatement = $con->prepare("FETCH 1 FROM cursor1");
while($innerStatement->execute() && $row = $innerStatement->fetch(PDO::FETCH_ASSOC)) {
echo $row['field'];
}
(The innerStatement->execute() needs calling on each loop iteration to move the cursor, and we also need to fetch the data from it …).
