uk.ac.starlink.ttools.cone
Class BlockUploader

java.lang.Object
  extended by uk.ac.starlink.ttools.cone.BlockUploader

public class BlockUploader
extends Object

Works with an UploadMatcher dividing the input table into chunks and uploading them separately to produce an arbitrarily large result while each upload/match operation is of a limited size.

Since:
15 May 2014
Author:
Mark Taylor

Constructor Summary
BlockUploader(UploadMatcher umatcher, int blocksize, long maxrec, String outName, JoinFixAction uploadFixAct, JoinFixAction remoteFixAct, ServiceFindMode serviceMode, boolean oneToOne, boolean uploadEmpty)
          Constructor.
 
Method Summary
 StarTable runMatch(StarTable inTable, QuerySequenceFactory qsFact, StoragePolicy storage)
          Performs an upload join in blocks.
 
Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
 

Constructor Detail

BlockUploader

public BlockUploader(UploadMatcher umatcher,
                     int blocksize,
                     long maxrec,
                     String outName,
                     JoinFixAction uploadFixAct,
                     JoinFixAction remoteFixAct,
                     ServiceFindMode serviceMode,
                     boolean oneToOne,
                     boolean uploadEmpty)
Constructor.

Parameters:
umatcher - upload matcher
blocksize - maximum number of rows per uploaded block
maxrec - maximum number of output rows accepted in total
outName - name of output table
uploadFixAct - name deduplication policy for upload table
remoteFixAct - name deduplication policy for remote table
serviceMode - upload match mode
oneToOne - true iff output rows match 1:1 with input rows
uploadEmpty - determines behaviour if there are no input rows: true means attempt the match anyway, false means throw an IOException
Method Detail

runMatch

public StarTable runMatch(StarTable inTable,
                          QuerySequenceFactory qsFact,
                          StoragePolicy storage)
                   throws IOException
Performs an upload join in blocks.

As currently implemented, the input table needs to have random access, and the output table has random access. It would be possible to stream on input and output to some extent, though each input chunk will still have to be random access.

The row indices given by ConeQueryRowSequence.getIndex calls associated with the supplied qsFact object must correspond to the row indices in the supplied inTable.

Parameters:
inTable - input table, must have random access
qsFact - object to generate positional queries when applied to a table
storage - storage policy for storing raw result table
Throws:
IOException


Copyright © 2015 Central Laboratory of the Research Councils. All Rights Reserved.