Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
GraphX benchmarking changes for VLDB
1. In-memory shuffle 2. Checking Iterator.hasNext 3. In-place/destructive updates in vertex replication 4. WikiPipelineBenchmark 5. K-Core 6. Dataflow (naive Spark) PageRank Squashed commit of the following: commit ad6590c Author: Ankur Dave <[email protected]> Date: Mon Apr 21 17:56:24 2014 -0700 Add unpersist option to Pregel commit 3bcaa2f Author: Dan Crankshaw <[email protected]> Date: Wed Apr 30 05:58:11 2014 +0000 Added faster version of kcore and fixed compile issue with dataflow pagerank commit 03f5d76 Author: Dan Crankshaw <[email protected]> Date: Wed Apr 30 02:22:13 2014 +0000 Added more connected components. commit cfd209b Author: Dan Crankshaw <[email protected]> Date: Wed Apr 30 00:01:24 2014 +0000 Initial CC dataflow implementation. commit 255db45 Author: Dan Crankshaw <[email protected]> Date: Tue Apr 29 19:38:26 2014 +0000 Updated pregel logging. commit 5032924 Author: Dan Crankshaw <[email protected]> Date: Tue Apr 29 08:52:07 2014 +0000 Added dataflow pagerank. Having issues with it and in-memory shuffle. commit f483ca4 Merge: 8d22500 9e59642 Author: Ankur Dave <[email protected]> Date: Tue Apr 29 00:05:49 2014 -0700 Merge pull request apache#3 from dcrankshaw/osdi_with_kcore_for_merge Osdi with kcore for merge commit 9e59642 Author: Dan Crankshaw <[email protected]> Date: Mon Apr 28 20:51:24 2014 +0000 Added more logging. Conflicts: graphx/src/main/scala/org/apache/spark/graphx/lib/Analytics.scala commit 76a6a54 Author: Dan Crankshaw <[email protected]> Date: Sun Apr 13 03:00:43 2014 +0000 Fixed kcore. now works. commit 1e924df Author: Dan Crankshaw <[email protected]> Date: Sun Apr 13 00:06:40 2014 +0000 Changed kcore filenames. commit a641dc1 Author: Dan Crankshaw <[email protected]> Date: Sat Apr 12 01:59:17 2014 +0000 Kcore debugging. commit dbe5180 Author: Dan Crankshaw <[email protected]> Date: Fri Apr 11 22:49:38 2014 +0000 Fixed results of cherry-pick commit 4ddc552 Author: Dan Crankshaw <[email protected]> Date: Tue Jan 7 13:44:46 2014 -0800 Added kcore implementation. Conflicts: graph/src/main/scala/org/apache/spark/graph/Pregel.scala commit 250199a Author: Dan Crankshaw <[email protected]> Date: Mon Apr 28 22:23:49 2014 +0000 Minor logging changes. commit 8d22500 Author: Ankur Dave <[email protected]> Date: Fri Apr 25 16:34:53 2014 -0700 EdgePartition.size should be val commit bb36cc8 Author: Ankur Dave <[email protected]> Date: Fri Apr 25 03:08:27 2014 -0700 Set locality wait commit ad4c874 Author: Ankur Dave <[email protected]> Date: Sat Apr 19 20:08:57 2014 -0700 In GraphLoader, coalesce to minEdgePartitions commit 2898126 Author: Ankur Dave <[email protected]> Date: Sat Apr 19 20:10:36 2014 -0700 In Analytics, take PageRank numIter commit 719b04a Author: Ankur Dave <[email protected]> Date: Sat Apr 19 20:09:24 2014 -0700 Log current Pregel iteration commit 8609184 Author: Ankur Dave <[email protected]> Date: Fri Mar 28 15:15:44 2014 -0700 Clean up WikiPipelineBenchmark commit e8be08e Author: Ankur Dave <[email protected]> Date: Fri Mar 28 15:14:50 2014 -0700 for-loop to while-loop in WikiArticle commit 540c267 Author: Ankur Dave <[email protected]> Date: Fri Mar 28 15:13:56 2014 -0700 Make mutable RDDs optional commit e27da44 Author: Ankur Dave <[email protected]> Date: Fri Mar 28 15:12:58 2014 -0700 Check for empty iterators commit 5ec645d Author: Ankur Dave <[email protected]> Date: Fri Mar 28 15:10:41 2014 -0700 In memory shuffle (cherry-picked from amplab/graphx#135) commit f36e576 Author: Ankur Dave <[email protected]> Date: Fri Mar 28 15:09:23 2014 -0700 Log warning on partition recompute commit c289e91 Author: Ankur Dave <[email protected]> Date: Fri Mar 28 15:08:06 2014 -0700 Untrack conf files commit a61be7a Author: Dan Crankshaw <[email protected]> Date: Tue Mar 25 07:03:41 2014 +0000 More timing logging and fixing the iterator issue. commit 0645183 Author: Dan Crankshaw <[email protected]> Date: Mon Mar 24 21:34:00 2014 +0000 Fixed IO bugs. commit a13f3b9 Author: Dan Crankshaw <[email protected]> Date: Mon Mar 24 19:52:46 2014 +0000 Fixed compile errors commit e6fc93b Author: Dan Crankshaw <[email protected]> Date: Mon Mar 24 03:29:19 2014 +0000 Updated pipeline benchmark to handle other systems. commit 0a04cb9 Author: Dan Crankshaw <[email protected]> Date: Sun Mar 23 20:57:08 2014 +0000 Renamed prepostprocess commit a262b07 Merge: 470a950 3342751 Author: Ankur Dave <[email protected]> Date: Fri Mar 21 11:39:02 2014 -0700 Merge remote-tracking branch 'dcrankshaw-incubator-spark/gx-vldb-bench' into vldb Conflicts: project/SparkBuild.scala commit 470a950 Merge: e09139d 3407a8f Author: Ankur Dave <[email protected]> Date: Fri Mar 21 11:30:12 2014 -0700 Merge branch 'mutable-rdd' into vldb commit 3342751 Author: Dan Crankshaw <[email protected]> Date: Sun Mar 2 02:14:46 2014 +0000 Fixed partitioning issues. commit 3407a8f Author: Ankur Dave <[email protected]> Date: Sat Mar 1 16:39:09 2014 -0800 Try mutating old RDDs for delta updates commit fad630f Author: Dan Crankshaw <[email protected]> Date: Wed Feb 26 02:33:01 2014 +0000 more debugging. Seems like we have empty partitions. commit b788bfb Author: Dan Crankshaw <[email protected]> Date: Tue Feb 25 11:10:16 2014 +0000 added more debug logging. commit 75b2da7 Author: Dan Crankshaw <[email protected]> Date: Tue Feb 25 09:47:32 2014 +0000 Fixed pipeline. Now seems to work well. commit 7799b3e Author: Dan Crankshaw <[email protected]> Date: Mon Feb 24 22:31:47 2014 +0000 updated slaves and conf. commit 95e19ea Author: Dan Crankshaw <[email protected]> Date: Sun Feb 23 06:19:21 2014 +0000 Readadded prepost commit da93886 Author: Dan Crankshaw <[email protected]> Date: Sun Feb 23 06:13:24 2014 +0000 Stringify still not working. FML. commit 6a4e492 Author: Dan Crankshaw <[email protected]> Date: Sun Feb 23 03:34:00 2014 +0000 Added debugging. Issue with stringify. commit 99d2713 Author: Dan Crankshaw <[email protected]> Date: Sat Feb 22 04:40:45 2014 +0000 Added new conf commit 7a036cb Author: Dan Crankshaw <[email protected]> Date: Fri Feb 21 20:35:37 2014 -0800 Fixed compilation issues. commit d3bbfd0 Author: Dan Crankshaw <[email protected]> Date: Wed Feb 19 06:15:14 2014 +0000 Beginning of new pipeline, not compiled yet. commit 2983132 Author: root <[email protected]> Date: Wed Feb 19 01:23:15 2014 +0000 Initial vldb benchmark commmit. Added slaves file.
- Loading branch information