从 std::list 中 size() 的时间复杂度引出的讨论...
File I/O 效率 C vs C++ (一)

fstream 文件 IO 点滴

輝夜(tadvent) posted @ 2009年10月06日 18:24 in C/C++ with tags VC STL c++ gcc string fstream , 4789 阅读

很多时候较大数据量的文件 IO 总是成为瓶颈,为了提高效率,有时想要先将文件大块大块的读入再行处理。下面分析两种惯常的处理手法。

1. 将文件一次性读入 string 中。

貌似 std::getline 、 istream::getline 或是 operator<< operator>> 等都不提供一次读到文件结尾的机制,只有 istreambuf_iterator 可以做到:

ifstream in("input.txt");
string instr((istreambuf_iterator<char>(in)), istreambuf_iterator<char>());

string 的构造函数前一个参数要多加一层 () 以免编译器误认为是函数声明 = = ...

这样读入 string 会随着内容动态增长,空间不足时会触发额外的 realloc 及 copy 操作,为提高效率有必要预分配足够的空间:

ifstream in("input.txt");
in.seekg(0, ios::end);
streampos len = in.tellg();
in.seekg(0, ios::beg);

string instr;
instr.reserve(len);
instr.assign(istreambuf_iterator<char>(in), istreambuf_iterator<char>());

2. 将文件一次性读入 stringstream 中。

filebuf 和 stringbuf 无法直接通过 rdbuf() 重定向,因此从 filebuf 到 stringbuf 需要一次 copy 操作。最简单的方法是直接复制整个 streambuf :

ifstream in("input.txt");
stringstream ss;
ss<<in.rdbuf();

与 string 的情况相同,这里同样也有一个空间 realloc 及 copy 的问题。但 streambuf 的缓冲区不是那么方便操作的,解决方法是我们给他手动指定一个空间:

ifstream in("input.txt");
in.seekg(0, ios::end);
streampos len = in.tellg();
in.seekg(0, ios::beg);

vector<char> buffer(len);
in.read(&buffer[0], len);

stringstream ss;
ss.rdbuf()->pubsetbuf(&buffer[0], len);

最后再顺便 BS 一下 VC 的 STL = =...

虽然 VC 的编译器效率没的说,但被 STL 拖后腿的话不就白搭了嘛。在文件 IO 方面 (fstream) 比起 MinGW (GCC 4.4.0) 带的要慢好几倍。GCC 的 fstream 格式化读写效率与 C 的比已经不分伯仲,以后应该还会有进一步的提升空间 (编译时格式控制 vs 执行时)

另外上面最后一段程序在 VS2008 (VC9.0) 下应该无法得到预想的结果,跟踪进去看了一下,VC 标准库里的 pubsetbuf 函数体居然是空的!内容如下(中间还有一层函数调用):

virtual _Myt *__CLR_OR_THIS_CALL setbuf(_Elem *, streamsize)
        {       // offer buffer to external agent (do nothing)
        return (this);
        }

看来是等着我们来继承了啊 = = 。而在 MinGW (GCC 4.4.0) 中可以得到预期的结果。

Nagaland 11th Questi 说:
2022年8月16日 21:29

The Nagaland 11th grade Important Question Paper 2023 was released in May of the previous year, and the exams were held in March. The 11th grade examinations were held in March this year, as they were the year before, and we anticipate that the May announcement of the Nagaland 11th Important Question Paper 2023 will be the same. Nagaland 11th Question Paper 2023 Therefore, following the release of the Nagaland 11th Model Paper, all students who took these examinations and are currently anticipating the NBSE +2 Important Question Paper 2023 of these exams may check their Nagaland 11th Important Question Paper 2023 by utilising the official web portal site of this board.

Emma 说:
2022年12月29日 19:54

In C++, the fstream file IO library provides various functions for reading from and writing to files. One of the most commonly used is the `getline` function, which reads a line of text from a file and stores it in a string Lab grown diamonds object. Other commonly used functions include `read`, `write`, and `seekg`, which allow you to read or write a certain number of bytes from a file, or seek to a specific position in a file, respectively.

seo service london 说:
2024年10月24日 15:13

that is an first rate detail for sharing this beneficial message. I'm dazzled by means of the facts you have got in this blog. It reasons me from more than one points of view. A debt of gratitude is in order for posting this another time. There's so much in this article that i would by no means have idea of by myself. Your content gives readers matters to consider in an thrilling way. Hi there! I ought to have sworn i’ve been to this weblog earlier than however after surfing via a number of the submit i found out it’s new to me. Anyhow, i’m without a doubt glad i found it and that i’ll be book-marking and checking back frequently

먹튀폴리스 说:
2024年10月24日 15:22

Thanks for all the tips mentioned in this article! it’s always good to read things you have heard before and are implementing, but from a different perspective, always pick up some extra bits of information.

카지노탐구생활 说:
2024年10月24日 15:25

Your article is very interesting. I think this article has a lot of information needed, looking forward to your new posts. Get permission to share .I bookmark this site and will track down your posts often from now on. Much obliged once more

get more info 说:
2024年10月24日 15:34

Youre so right. Im there with you. Your weblog is definitely worth a read if anyone comes throughout it. Im lucky I did because now Ive received a whole new view of this.

website 说:
2024年10月24日 15:38

 Exceptional examine, positive website online, in which did u come up with the data on this posting? I have read a number of the articles in your website now, and i definitely like your style. Thank you a million and please maintain up the powerful work.

website 说:
2024年10月24日 15:42

 fascinating dialog is value remark. I feel that it is best to compose more on this matter, it may not be an unthinkable theme however generally people are insufficient to chat on such subjects

reference material 说:
2024年10月24日 15:46

Are you playing in an old, beat up, pair of basketball shoes? You could be risking injury, poor game play, and more! This article shows you why picking the right basketball is so important, and how to do it properly. 

read more 说:
2024年10月24日 15:49

Positive site, where did u come up with the iinformation on this posting?I have read a few of the articles on your website now, and I really like your style. Thanks a million and please keep up the effective work. Feel free to visit my website

click here 说:
2024年10月24日 15:56

When I read your article on this topic, the first thought seems profound and difficult. There is also a bulletin board for discussion of articles and photos similar to this topic on my site, but I would like to visit once when I have time to discuss this topic

contents 说:
2024年10月24日 16:05

I found this is an informative and interesting post so i think so it is very useful and knowledgeable. I would like to thank you for the efforts you have made in writing this article.

casinosesang 说:
2024年10月24日 16:21

Magnificent items from you, man. I have consider your stuff prior to and you’re. just too excellent. I actually like what you have bought right here,I read this post fully regarding the difference of most recent and preceding technologies, it’s awesome article.I cant wait to learn far more from you. That is actually a terrific website. 

information 说:
2024年10月24日 16:29

Thanks for all the tips mentioned in this article! it’s always good to read things you have heard before and are implementing, but from a different perspective, always pick up some extra bits of information.

먹튀서포트 说:
2024年10月24日 16:33

’d really love to be a part of online community where I can get responses from other knowledgeable individuals that share the same interest. If you have any recommendations, please let me know. Appreciate it! Feel free to visit my website;

here 说:
2024年10月24日 16:33

I found this is an informative and interesting post so i think so it is very useful and knowledgeable. I would like to thank you for the efforts you have made in writing this article.

website 说:
2024年10月24日 16:37

Thanks for all the tips mentioned in this article! it’s always good to read things you have heard before and are implementing, but from a different perspective, always pick up some extra bits of information.

check here 说:
2024年10月24日 16:43

I think commenting is the best part of my blogging – especially here at Pro Blogger. You see I’m not that profound or wise, but many of my readers are. Comments add value to my blog. They take my posts to the next level and often take my ideas in rewarding new directions.”

먹튀검증 说:
2024年10月24日 16:43

When I originally commented I clicked the -Notify me when new surveys are added- checkbox now whenever a comment is added I get four emails with the same comment. Could there be however you may get rid of me from that service? Thanks!

good contents 说:
2024年10月24日 16:45

’d really love to be a part of online community where I can get responses from other knowledgeable individuals that share the same interest. If you have any recommendations, please let me know. Appreciate it! Feel free to visit my website;

References 说:
2024年10月24日 16:52

Excellent read, I just passed this onto a friend who was doing a little research on that. And he actually bought me lunch as I found it for him smile Therefore let me rephrase that: Thank you for lunch.

information 说:
2024年10月24日 17:04

Your article is very interesting. I think this article has a lot of information needed, looking forward to your new posts. Get permission to share .I bookmark this site and will track down your posts often from now on. Much obliged once more

good information 说:
2024年10月24日 17:18

Wonderful post! We are linking to this great post on our website. Keep up the good writing. pretty handy stuff, overall I imagine this is really worth a bookmark, thanks Feel free to visit my website

View data 说:
2024年10月24日 17:34

While looking for articles on these topics, I came across this article on the site here. As I read your article, I felt like an expert in this field. I have several articles on these topics posted on my site

reference material 说:
2024年10月24日 17:55

I know that you explain it very well. And I hope that other readers will also experience how I feel after reading your article. I really enjoy simply reading all of your weblogs. Simply wanted to inform you that you have people like me who appreciate your work. Definitely a great post. Hats off to you! The information that you have provided is very helpful. 


登录 *


loading captcha image...
(输入验证码)
or Ctrl+Enter