python pandas 怎样高效地添加一行数据

That's probably as efficient as any, but Pandas/numpy

structures are fundamentally not suited for efficiently growing. They

work best when they are created with a fixed size and stay that way. – BrenBarnDec 6 '12 at 20:43

append

is a wrapper for concat, so concat would be marginally more efficient,

but as @BrenBarn says Pandas is probably not appropriate for updating a

HDF5 file every second. If you absolutely need Pandas for some reason, could you collect a list of Series and update the file periodically instead? – Matti JohnDec 6 '12 at 20:54

Bren is right about numpy/pandas working best when preallocated. If memory is no constraint just preallocate a huge zeros array and append at the end of the program removing any excess zeros. Which I suppose is a bit of what Matti is saying. – arynaqDec 6 '12 at 21:16Intro to Data Structures

DataFrame is a 2-dimensional labeled data structure with columns of potentially different types.

所

以一般说来dataframe就是a set of columns, each column is an array of values. In

pandas, the array is one way or another a (maybe variant of) numpy

ndarray. 而ndarray本身不存在一种in place

append的 *** 作。。。因为它实际上是一段连续内存。。。任何需要改变ndarray长度的 *** 作都涉及分配一段长度合适的新的内存，然后copy。。。

这是这类 *** 作慢的原因。。。如果pandas dataframe没有用其他设计减少copy的话，我相信Bren说的"That's probably

as efficient as any"是很对的。。。

所以in general, 正如Bren说的。。。Pandas/numpy structures are fundamentally not suited for efficiently growing.

Matti 和 arynaq说的是两种常见的对付这个问题的方法。。。我想Matti实际的意思是把要加的rows收集成起来然后concatenate, 这样只copy一次。arynaq的方法就是预先分配内存比较好理解。。。

如果你真的需要incrementally build a dataframe的话，估计你需要实际测试一下两种方法。。。

我的建议是，如有可能，尽力避免incrementally build a dataframe, 比如用其他data structure 收集齐所有data然后转变成dataframe做分析。。。

顺便。。。这类问题上stackoverflow好得多。。

导入 基本 *** 作 shape值 array数组 基本 *** 作 导入 基本 *** 作 对一个列表进行加一 *** 作报错，列表没有这个 *** 作,当使用numpy时，再执行该 *** 作，就不会报错，numpy的底层数据结构为numpy.ndarray结构 同时array数组是可以和array数组进行基本的算数 *** 作的，但是shape值(每一个数组的元素个数)必须相同，取数据取索引就可以 shape值 可以...

---