Posted on

Since Microsoft introduced Shredded Storage for SharePoint 2013, several myths came about concerning the guidelines and utilisation of the feature in conjunction with Remote BLOB Storage (RBS). According to this, I already heard that RBS is really a synonym of Shredded Storage. Furthermore, I had been facing the assumptions that because of small file chunk size, the RBS threshold should never be triggered which Shredded Storage even makes RBS obsolete.

In the following paragraphs, I’ll clarify the reality behind the parable and show best practice strategies for how you can configure Shredded Storage, RBS, as well as both features together.

Because the name states, RBS externalizes Binary Large Objects (BLOB). BLOBs are often stored being an “image-file” in SQL content database having a size limit of two GB. Since Exterior BLOB Storage (EBS) and finally RBS were introduced, these files are held in a Varbinary format. This extendable doesn’t really need a size limit, but Microsoft is constantly on the enforce this 2 GB limitation in SharePoint for performance reasons.

Reducing database size through externalization produces a better performance. It was a significant objective of RBS since the SQL storage is extremely costly as well as very resource-intensive. Thus, it may be beneficial to externalize large files into cheaper SAN, NAS, or any other storage devices. Prior to the implementation of Shredded Storage, using RBS to externalize files bigger than 1 MB labored well for a lot of organizations. However, Shredded Storage introduced newer and more effective and improved features.

When developing the Shredded Storage, Microsoft centered on these primary goals:

  1. Reduce Storage
    • With Shredded Storage, SharePoint stores only changes to some file, and not the entire file.
  2. Optimize Bandwidth
    • Only parts need to be sent. This enables multi-user functionality, like co-authoring on Office documents (set one part to see-only, allowing other areas to become altered and rapidly synchronized)
  3. Optimize File I/O
    • Smoother I/O Patterns
    • Ensure write pricing is proportional to size change
    • Put I/Ops from WFE to SQL and just changes create information in transaction logs (this even improves backup performance)
  4. Security
    • It is a lot more nearly impossible to find personal files from the database using scripts.
    • Each chunk is encrypted individually.

Basically, it’s the splitting of the file into several small pieces known as “chunks.” The “FileWriteChunkSize” defines how big the file pieces, while adding them into SQL Server. However, not every chunks are identical size. For instance, a thing document’s header and footer may well be a different size compared to file’s other chunks. It is because more details is needed to rebuild the whole file of all the small pieces. However, when the first changes are created to personal files with versioning settings enabled, the first bigger size is going to be paid for since the alterations have to be saved.

According to several tests with various files sizes, a 1250 KB FileWriteChunkSize was recognized as a finest practice since it enables the quickest upload and download performance. Based on your atmosphere in addition to size and type of the data, a bigger shred size might be possible. Example: the Microsoft Cloud storage system OneDrive for Business utilizes a 2 MB shred size. (Further independent test is a result of Microsoft MVP Chris McNulty).

When it comes to storage optimization, exactly what does RBS provide for you? The primary objective of RBS may be the externalization of files according to size or any other qualities. Poor SharePoint’s limitations (for example, the 200 GB limit for content databases), externalized data still counts. As the row entry within the SQL content database table (dbo.AllDocs) remains, the file is taken off the database file and gone to live in a configured storage.

Shredded Storage and RBS - SharePoint Storage Optimization

Whenever you let the RBS provider per database, RBS system tables will be included to the information database. During these tables, mainly in the mssqlrbs_sources.rbs_internal_blobs table, are where externalized data pieces are listed. Whenever a user accesses personal files, the RBS provider retrieves the metadata in the dbo.AllDocs table. It will likewise search for the externalized file using a mapped SharePoint Globally Unique Identifier (GUID) within the RBS related table and cargo then your file in the referenced storage location.

Even though the externalized files may also be considered within the database size, it is simple to attain the needs for longer content databases. For instance, Microsoft supports (archive) content databases with as many as 4 TB on the disk subsystem having a minimum .25 I/Ops per GB. Because of the externalization, there isn’t any real 4 TB database file. There’s maybe only 150 GB, so that you can easily fulfill the high subsystem needs with standard drives (suggested 2 I/Ops per GB).

Example calculation:

Effective data in der database Needed I/Ops for that Disk Subsystem
4 TB     (à ,25 I/Ops * 1024 * 4) 1024
150 GB (à ,25 I/Ops * 150) 37.5


If you are planning to realize this and extend past the 200 GB threshold “virtually” with RBS, it’s not a problem. However, take into account that there might be situations where the externalized data needs to return in to the database, e.g. for any migration, or maybe externalized files need to be incorporated inside your backup strategy. Solutions like AvePoint’s DocAve Software might help during these scenarios.

While externalization may be the primary objective of RBS, there’s also extra goals, which may be achieved:

  1. Performance

The RBS framework offers a large performance gain. It is because the SQL formula is made for small data pieces across tables, although not for big objects. Even though it sounds weird to load just the metadata in the content databases and still provide the file itself from the connected storage, the RBS approach is actually considerably faster when compared with loading altogether from SQL.

This relies upon the subsystem for SQL, but in addition for RBS storage. The network performance may also influence the loading speed when the storage is connected differently than SQL. To be able to load the BLOB in the storage and render it for that user, the browser must complete the next steps:

  1. DNS Lookup
  2. Initial Connection
  3. Waiting
  4. Receive data
  5. Close connection

The awaiting information is known as TTFB (Time for you to First Byte). 100-200 ms are usually good values. However, if you are planning for connecting a NAS (Network Attached Storage) as RBS storage, Microsoft needs a TTFB of under 20 ms.

  1. Deduplication

An additional advantage of utilizing RBS is you can utilize deduplication around the storage. What this means is, for those who have stored exactly the same document in a variety of libraries, the deduplication formula will identify it, store just one replica, but write a catalog which locations the file was discovered. This protects additional space. As opposed to the scenario with Shredded Storage, once the same document is kept in several libraries, it it’s still stored several occasions within the database.

Several tests demonstrated the very best achievable performance by having an RBS threshold of 1024 KB. However, the independent is a result of Chris McNulty also explain, this value can vary when it comes to quality and atmosphere. According to this, you should know the RBS threshold needs to be smaller sized compared to FileWriteChunkSize, otherwise the externalization can’t be triggered. This is applicable towards the real-time externalization. However, because the real-time processing is extremely resource hungry, Microsoft is not recommending this. Rather, solutions like DocAve Storage Manager are a good alternative. DocAve doesn’t think about the chunk size – it concentrates on the actual size the whole file. Additionally, further filters could be configured, for example externalization according to document version, last access time, custom posts, and document qualities.

Shredded Storage and RBS – Better Together

Since we’ve identified their variations, how will you use RBS in conjunction with Shredded Storage to make the most of the 2 features? So that you can answer this, we have to identify whether both technology is needed. The next questions can help determine the solution:

Shredded Storage? – Answer with Yes:

  1. Exist many (concurrent) transactions?
  2. Would you use versioning?
  3. Have you got many Office files?
  4. Would you like to use co-authoring?
  5. Have you got I/Ops and network issues?

RBS? – Answer with Yes:

  1. Have you got many large files (&gt1MB)?
  2. Would you use versioning on individuals files?
  3. Have you got a SQL storage issue? (Content database bigger than 100GB)
  4. Must you accelerate Content database backups?
  5. Have you got exactly the same files submitted to various libraries and wish to use deduplication?

Based on your solutions, you might choose one, another, or both technologies. In case your company may benefit from both approaches, you won’t just realize the benefits of each technology, but additionally achieve yet another performance boost. Should you configured the 2 concepts using the pointed out guidelines, you are able to achieve loading occasions as much as 80 % faster (see results from Microsoft MVP Chris McNulty).

May be the RBS technology obsolete if you have Shredded Storage in position? NO! Nonetheless, your SharePoint atmosphere, its problems, as well as your business goals determines which mixture of technologies brings the greatest benefit for the company.

Try DocAve Free for 30 Days