Esta página aún no se ha traducido para esta versión. Puede ver la versión más reciente de esta página en inglés.

MapReduce

Técnica de programación para analizar conjuntos de datos que no caben en la memoria

mapreduce es una técnica de programación adecuada para analizar conjuntos de datos de gran tamaño que, de lo contrario, no cabrían en la memoria de su equipo. Utilizando datastore para procesar los datos en pequeñas partes, la técnica consta de una fase de asignación, donde se da formato a los datos o se realiza un cálculo precursor, y una fase de reducción, donde se agregan todos los resultados provenientes de la fase de asignación. Para obtener más información, consulte Getting Started with MapReduce.

Para obtener información sobre cómo usar otros productos con mapreduce, consulte Speed Up and Deploy MapReduce Using Other Products.

Funciones

expandir todo

General

`mapreduce`	Programming technique for analyzing data sets that do not fit in memory
`datastore`	Create datastore for large collections of data

Métodos de objeto auxiliar

`add`	Add single key-value pair to KeyValueStore
`addmulti`	Add multiple key-value pairs to KeyValueStore
`hasnext`	Determine if ValueIterator has one or more values available
`getnext`	Get next value from ValueIterator

Entorno de ejecución

`mapreducer`	Define execution environment for mapreduce or tall arrays
`gcmr`	Get current mapreducer configuration

Objetos

`KeyValueStore`	Store key-value pairs for use with mapreduce
`ValueIterator`	An iterator over intermediate values for use with mapreduce

Temas

Getting Started with MapReduce

Learn about the MapReduce programming technique and run an example calculation.

Write a Map Function

Create a map function for use in a mapreduce algorithm.

Write a Reduce Function

Create a reduce function for use in a mapreduce algorithm.

Build Effective Algorithms with MapReduce

Summary of mapreduce example files.

Speed Up and Deploy MapReduce Using Other Products

Capabilities of other products to speed up and share mapreduce algorithms.

Find Maximum Value with MapReduce

This example shows how to find the maximum value of a single variable in a data set using mapreduce.

Compute Mean Value with MapReduce

This example shows how to compute the mean of a single variable in a data set using mapreduce.

Create Histograms Using MapReduce

This example shows how to visualize patterns in a large data set without having to load all of the observations into memory simultaneously.

Compute Mean by Group Using MapReduce

This example shows how to compute the mean by group in a data set using mapreduce.

Simple Data Subsetting Using MapReduce

This example shows how to extract a subset of a large data set.

Using MapReduce to Compute Covariance and Related Quantities

This example shows how to compute the mean and covariance for several variables in a large data set using mapreduce.

Compute Summary Statistics by Group Using MapReduce

This example shows how to compute summary statistics organized by group using mapreduce.

Using MapReduce to Fit a Logistic Regression Model

This example shows how to use mapreduce to carry out simple logistic regression using a single predictor.

Tall Skinny QR (TSQR) Matrix Factorization Using MapReduce

This example shows how to compute a tall skinny QR (TSQR) factorization using mapreduce.

Compute Maximum Average HSV of Images with MapReduce

This example shows how to use ImageDatastore and mapreduce to find images with maximum hue, saturation and brightness values in an image collection.

Solución de problemas

Debug MapReduce Algorithms

This example shows how to debug your mapreduce algorithms in MATLAB^® using a simple example file, MaxMapReduceExample.m. Debugging enables you to follow the movement of data between the different phases of mapreduce execution and inspect the state of all intermediate variables.

Ejemplos destacados

Analyze Big Data in MATLAB Using MapReduce

Use the mapreduce function to process a large amount of file-based data. The MapReduce algorithm is a mainstay of many modern "big data" applications. This example operates on a single computer, but the code can scale up to use Hadoop®.

Abrir script en vivo