HDInsightクラスターで毎日のWebトラフィックログの分析を実行するプロセスを管理します。 250台のWebサーバーはそれぞれ、毎日約10メガバイト(MB)のログデータを生成します。すべてのログデータは、Microsoft Azure Data Lake Storage Gen2の単一のフォルダーに保存されます。 プロセスのパフォーマンスを改善する必要があります。 どの2つの変更を行う必要がありますか?それぞれの正解は完全な解決策を提示します。 注:正しい選択はそれぞれ1ポイントの価値があります。
正解:A,C
Explanation A: Typically, analytics engines such as HDInsight and Azure Data Lake Analytics has a per-five overhead. If you store your data as many small files, this can negatively affect performance. In general, organize your data into larger sized files for better performance (256MB to 100GB in size). Some engines and applications might have trouble efficiently processing files that are greater than 100GB in size. C: For Hive workloads, partition pruning of time-series data can help some queries read only a subset of the data which improves performance. Those pipelines that ingest time-series data, often place their files with a very structured naming for files and folders. Below is a very common example we see for data is structured by date: \DataSet\YYYY\MM\DD\datafile_YYYY_MM_DD.tsv Notice that the datetime information appears both as folders and in the filename. Reference: https://docs.microsoft.com/en-us/azure/storage/blobs/data-lake-storage-performance-tuning-guidance